pytorch predictions different between train and predict

A

Amw 5G

Guest
Amw 5G Asks: pytorch predictions different between train and predict
I am training a pytorch model, which is itself an attribute of larger class containing the trained model plus other objects needed for inference and diagnostics. The predictions from the final epoch of my training are not matching the predictions from inference, given the same inputs.

I define a torch model class, something like this:

Code:
class torch_model_basic(nn.Module):
    def __init__(self, some_args):
        ...
    def forward(self, some_args):
        ....
    return x

My custom class that uses the torch model class looks something like this:

Code:
class MyClass(object):
    def __init__(self, config_file_path):
        self.device = 'cuda' if torch.cude.is_available() else 'cpu'
        ...

    def make_other_attribs(self, more_args):
        ...
        return more_attribs

    def train_one_epoch(self, my_model, optimizer, some_other_args):
        ...
        return model, training_input_data, training_predictions

    def train(self, args, torch_data_set):
        model = torch_model_basic(args=args) # see how we use it?
        optimizer = ...
        for epoch in args.epochs:
            model, inputs, predictions = self.train_one_epoch(model, optimizer, some_other_args)
        self.model = model.to('cpu') # make the trained model an attribute of our larger object
        self.training_predictions = predictions # so I can look at the predictions after training
        self.more_attribs = self.make_other_attribs() # lots of other stuff happens here, unrelated to training
        return self

    def predict(self, args, torch_data_set):
        model = self.model
        model.to_device(self.device).eval()
        ... #build the x_pred inference tensor
        with torch.no_grad():
            preds = model(x_pred)
        return preds

To train & save my object, I do something like this:

Code:
my_object = MyClass()
my_dataset = ... # build the training data
my_model = my_object.train(args, my_dataset)
with open('file/path/model.pickle', 'wb') as handle:
    pickle.dump(my_model, handle, protocol=pickle.HIGHEST_PROTOCOL)

To predict, it's very similar:

Code:
with open('file/path/model.pickle', 'rb') as handle:
    my_model = pickle.load(handle)
my_dataset = ... # build the prediction data, which is the same as the training data
new_predictions = my_model.predict(args, my_dataset)

I have a bunch of print statements to watch the data as it gets read in, gets transformed, gets fed to the model, and the resulting predictions. I do this for both training & prediction.

After the model has been trained, I then feed the training data back into the prediction method. I can confirm the step immediately before pred = model(x_pred) has a tensor that duplicates the training data. But the predictions return from the prediction call do not match the predictions that I persist from the train() call.

I am training on a GPU and inferring on a CPU, if that matters. And I am using AWS SageMaker to do the training/inference on remote containers. Which, by the way, makes debugging super tough. The training call saves object to S3; the prediction routine picks up the saved object from S3 in the same place.

Any obvious reason why I shouldn't expect the predictions during training (from the final epoch) to be the same as the predictions during inference to be the same, given equivalent input data? I'm new to OOP, so pointers on why what I'm doing is a bad thing are also welcome.

SolveForum.com may not be responsible for the answers or solutions given to any question asked by the users. All Answers or responses are user generated answers and we do not have proof of its validity or correctness. Please vote for the answer that helped you in order to help others find out which is the most helpful answer. Questions labeled as solved may be solved or may not be solved depending on the type of question and the date posted for some posts may be scheduled to be deleted periodically. Do not hesitate to share your thoughts here to help others.
 
/forums/whats-new/news-feed

Unreplied Posts

Debug assertion failed when trying to delete array from heap

Hofbr Asks: Debug assertion failed when trying to delete array from heap
I have a sub-optimal solution to return two variables from a function by creating a pointer to an array with two indexes. (just using my current knowledge base from class. I know vectors or std::pair would be a better solution)

However, when I try to delete the array via the pointer, I'm getting the below error.

Am I misunderstanding the procedure? Don't know I need to delete the array created in heap?

enter image description here

Code:
#include <iostream>
#include <cmath>

double* quadraticEquation(double a, double b, double c) {
    double x1, x2;

    x1 = (-b + sqrt(b * b - 4 * a * c)) / (2 * a);
    x2 = (-b - sqrt(b * b - 4 * a * c)) / (2 * a);
    double arr[2] = {x1, x2};
    return arr;
}

int main()
{
    double* solution = quadraticEquation(3, 5, -8);

    double x1 = solution[0];
    double x2 = solution[1];
    std::cout << "x1 = " << x1 << std::endl << "x2 = " << x2 << std::endl;

    delete[] solution;
    solution = nullptr;
}

SolveForum.com may not be responsible for the answers or solutions given to any question asked by the users. All Answers or responses are user generated answers and we do not have proof of its validity or correctness. Please vote for the answer that helped you in order to help others find out which is the most helpful answer. Questions labeled as solved may be solved or may not be solved depending on the type of question and the date posted for some posts may be scheduled to be deleted periodically. Do not hesitate to share your thoughts here to help others.

Apache - I can open every file except the index

Doctore Asks: Apache - I can open every file except the index
Fresh Apache install on Fedora Server.

After replacing the default index.html with one of my own (just a boilerplate html with "Test" in the body) - I am unable to open it. I get Forbidden/You don't have permission to access this resource.

The real kicker is, I have two other files in the /var/www/html folder, one html the other php, and I can open both of them.

Apache is listening on port 80 and the port is open in the firewall. Permissions for all 3 files are exactly the same, along with owner/group.

Btw, I can't access the index.html either from the browser of another computer on the same network, or directly at the server cli with wget.

Any ideas what's going on here?

SolveForum.com may not be responsible for the answers or solutions given to any question asked by the users. All Answers or responses are user generated answers and we do not have proof of its validity or correctness. Please vote for the answer that helped you in order to help others find out which is the most helpful answer. Questions labeled as solved may be solved or may not be solved depending on the type of question and the date posted for some posts may be scheduled to be deleted periodically. Do not hesitate to share your thoughts here to help others.

Using MySql for a backend for flutter app?

Dumb Nerd Asks: Using MySql for a backend for flutter app?
Been doing some deep diving in regards to building a backend for my flutter application. The premise and requirements I am looking for are something that is self-hosted and ultimately free (no monthly charges, overuse, etc like firebase) only the essentials such as storage that I would pay for IF AND ONLY IF required; in other words, it's my responsibility. I just downloaded MySql, my only question is how should I go about what APIs I could use for caching and passing data to the application, or do I even need a middleman to pass data? Input is much appreciated.

SolveForum.com may not be responsible for the answers or solutions given to any question asked by the users. All Answers or responses are user generated answers and we do not have proof of its validity or correctness. Please vote for the answer that helped you in order to help others find out which is the most helpful answer. Questions labeled as solved may be solved or may not be solved depending on the type of question and the date posted for some posts may be scheduled to be deleted periodically. Do not hesitate to share your thoughts here to help others.

XCode 14 SwiftUI dyld symbol not found

Deepak Sharma Asks: XCode 14 SwiftUI dyld symbol not found
I get this error when I run my code on an iOS 14 device using XCode 14.

Code:
  dyld: Symbol not found: _$s7SwiftUI15GraphicsContextV4fill_4with5styleyAA4PathV_AC7ShadingVAA9FillStyleVtF
Referenced from: ******
Expected in: /System/Library/Frameworks/SwiftUI.framework/SwiftUI

I do not get any errors when I run the same code on an iOS 16 device.

How do I resolve this issue?

SolveForum.com may not be responsible for the answers or solutions given to any question asked by the users. All Answers or responses are user generated answers and we do not have proof of its validity or correctness. Please vote for the answer that helped you in order to help others find out which is the most helpful answer. Questions labeled as solved may be solved or may not be solved depending on the type of question and the date posted for some posts may be scheduled to be deleted periodically. Do not hesitate to share your thoughts here to help others.

How do I resolve Java Eclipse command errors that do not meet the conditions?

Soohyun Kang Asks: How do I resolve Java Eclipse command errors that do not meet the conditions?

It works well if I type the word that has more than 6 characters in the console, but if I type the word that has less than 6 characters, an error appears as shown in the picture. In this case, how do I solve this problum on words with less than 6 characters?

SolveForum.com may not be responsible for the answers or solutions given to any question asked by the users. All Answers or responses are user generated answers and we do not have proof of its validity or correctness. Please vote for the answer that helped you in order to help others find out which is the most helpful answer. Questions labeled as solved may be solved or may not be solved depending on the type of question and the date posted for some posts may be scheduled to be deleted periodically. Do not hesitate to share your thoughts here to help others.

Latest posts

Top