Can I minimize a mysterious function by running a gradient decent on her neural net approximations? [closed]

V

Vladimir Zolotov

Guest
Vladimir Zolotov Asks: Can I minimize a mysterious function by running a gradient decent on her neural net approximations? [closed]
A cross post from on AI StackExchange, and MathOverflow

So I have this function let call her $F:[0,1]^n \rightarrow \mathbb{R}$ and say $10 \le n \le 100$. I want to find some $x_0 \in [0,1]^n$ such that $F(x_0)$ is as small as possible. I don't think there is any hope of getting the global minimum. I just want a reasonably good $x_0$.

AFAIK the standard approach is to run an (accelerated) gradient decent a bunch of times and take the best result. But in my case values of $F$ are computed algorithmically and I don't have a way to compute gradients for $F$.

So I want to do something like this.

(A) We create a neural network which takes an $n$-dimensional vector as input and returns a real number as result. We want the NN to "predict" values of $F$ but at this point it is untrained.

(B) We take bunch of random points in $[0,1]^n$. We compute values of $F$ at those points. And we train NN using this data.

(C1) Now the neural net provides us with a reasonably smooth function $F_1:[0,1]^n \rightarrow \mathbb{R}$ approximating $F$. We run a gradient decent a bunch of times on $F_1$. We take the final points of those decent and compute $F$ on them to see if we caught any small values. Then we take whole paths of those gradient decent, compute $F$ on them and use this as data to retrain our neural net.

(C2) The retrained neural net provides us with a new function $F_2$ and we repeat the previous step

(C3) ...

Does this approach have a name? Is it used somewhere? Should I indeed use neural nets or there are better ways of constructing smooth approximations for my needs?

SolveForum.com may not be responsible for the answers or solutions given to any question asked by the users. All Answers or responses are user generated answers and we do not have proof of its validity or correctness. Please vote for the answer that helped you in order to help others find out which is the most helpful answer. Questions labeled as solved may be solved or may not be solved depending on the type of question and the date posted for some posts may be scheduled to be deleted periodically. Do not hesitate to share your thoughts here to help others.
 
/forums/whats-new/news-feed

Unreplied Posts

Angular Material Ripple Effect is off to side of element

bokobaba Asks: Angular Material Ripple Effect is off to side of element
I have added angular material to my project and set up an empty component with the html from the tabs example.

Code:
<mat-tab-group>
    <mat-tab label="First"> Content 1 </mat-tab>
    <mat-tab label="Second"> Content 2 </mat-tab>
    <mat-tab label="Third"> Content 3 </mat-tab>
</mat-tab-group>

Every time I click on a tab or even if I add a button the ripple effect shows outside the element off to the side. Here's a picture of what is happening.
SolveForum.com may not be responsible for the answers or solutions given to any question asked by the users. All Answers or responses are user generated answers and we do not have proof of its validity or correctness. Please vote for the answer that helped you in order to help others find out which is the most helpful answer. Questions labeled as solved may be solved or may not be solved depending on the type of question and the date posted for some posts may be scheduled to be deleted periodically. Do not hesitate to share your thoughts here to help others.

cordova DeprecationWarning: OutgoingMessage.prototype._headers is deprecated

Khalil Baba Asks: cordova DeprecationWarning: OutgoingMessage.prototype._headers is deprecated
I am working on a cordova app. I have successfully built a static app which i can test on a mobile device. After installing the "nodejs-mobile-cordova", i get a successful bridge. But i get an error when trying to run the server by adding the source to the main initiation index.js file which doesn't respond after it runs.

This is the errorenter image description here

SolveForum.com may not be responsible for the answers or solutions given to any question asked by the users. All Answers or responses are user generated answers and we do not have proof of its validity or correctness. Please vote for the answer that helped you in order to help others find out which is the most helpful answer. Questions labeled as solved may be solved or may not be solved depending on the type of question and the date posted for some posts may be scheduled to be deleted periodically. Do not hesitate to share your thoughts here to help others.

I am trying to solve the supervsied learning problem (housing price) using pytorch and I encounter CUDA error?

Manish Kumar Singh Asks: I am trying to solve the supervsied learning problem (housing price) using pytorch and I encounter CUDA error?
I am getting the following error:

Code:
    ---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
/tmp/ipykernel_17/1714279870.py in <module>
      4 
      5 
----> 6 data_tensor = CustomData(data)
      7 model = NeuralNetwork().to(device)
      8 train_data_tensor = DataLoader(data_tensor,batch_size,shuffle=True)

/tmp/ipykernel_17/3667328859.py in __init__(self, data, transform, target_transform)
     33         #self.data = data
     34         self.label = data[:,-1]
---> 35         self.label = self.label.to(device)
     36 
     37         self.feature = data[:,:-1]

RuntimeError: CUDA error: device-side assert triggered
CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.

My code is

Code:
import numpy as np 
import pandas as pd 
import matplotlib.pyplot as plt
from pandas import DataFrame
from torch.utils.data import Dataset
import torch

import torch.nn as nn
from torch.utils.data import DataLoader
import torchvision
from torchvision import datasets, transforms
import torch.optim as optim
from torch.autograd import Variable
import torch.nn.functional as F
import os
for dirname, _, filenames in os.walk('/kaggle/input'):
    for filename in filenames:
        print(os.path.join(dirname, filename))

Here I try fill gaps in the data and also convert text data (like good, and bad) into numbers and finally convert the entire thing into a tensor. So far no error.

Code:
data = pd.read_csv('../input/house-prices-advanced-regression-techniques/train.csv')
data_cat = data[[str(i) for i in data.columns if data[str(i)].dtype == object]]
data_cat = data_cat.rank(axis=0, method='dense', na_option='top', ascending=False).astype(float)#converting text data to numbers
data_cat.fillna(0)
data_num = data[[str(i) for i in data.columns if data[str(i)].dtype != object]]
data_val = data_num.iloc[:,-1]
data_val = data_val.fillna(0)
data_num = data_num.iloc[:,:-1]
data_num = (data_num - data_num.mean())/(data_num.std())
data_num = data_num.fillna(0)
data = pd.concat([data_cat,data_num,data_val],axis=1)
data = torch.from_numpy(data.to_numpy())

Now I create a dataset to be used in NeuralNetwork via Dataloader. Again so far so good.

Code:
class CustomDataset(Dataset):
    def __init__(self,feature, label, transform=None, target_transform=None):
        self.feature = feature
        self.label = label
        self.transform = transform
        self.target_transform = target_transform
    def __len__(self):
        return len(self.label)
    def __getitem__(self, idx):
        feature = self.feature[idx]
        label = self.label[idx]
        if self.transform:
            feature = self.transform(feature)
        if self.target_transform:
            label = self.target_transform(label)
        return feature, label      
data_tensor = CustomDataset(data[:,:-1],data[:,-1])

Since I will be using GPU, I define the device.

Code:
device = "cuda" if torch.cuda.is_available() else "cpu"
print(f"Using {device} device")

This is my nn model. Because there are categorical data, I am using an embedding layer first.

Code:
class NeuralNetwork(nn.Module):
    def __init__(self):
        super(NeuralNetwork, self).__init__()
        
        self.linear_relu_stack = nn.Sequential(
            nn.Embedding(79,10),
            nn.Linear(70, 10),
            nn.ReLU(),
            nn.Linear(10, 10),
            nn.ReLU(),
            nn.Linear(10, 10),
            nn.ReLU(),            
            nn.Linear(10, 1),
        )
    def forward(self, x):
        logits = self.linear_relu_stack(x)
        return logits

These are my hyperprameters.

Code:
learning_rate = 0.005
batch_size = 20
num_of_epoch = 50

Now I am setting up the training process. I suspect this is where the bug lies.

Code:
model = NeuralNetwork().to(device)
train_data_tensor = DataLoader(data_tensor,batch_size,shuffle=True)
loss_fn = nn.MSELoss()
optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)
def train_loop(data_loader, model, loss_fn, optimizer):
    size = len(data_loader.dataset)
    for batch, (X, y) in enumerate(data_loader):
        # Compute prediction and loss
        pred = model(X)
        loss = loss_fn(pred, y)
        # Backpropagation
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()        
        if batch % 100 == 0:
            loss, current = loss.item(), batch * len(X)
            print(f"loss: {loss:>7f}  [{current:>5d}/{size:>5d}]")

#Actual training
for t in range(num_of_epoch):
    print(f"Epoch {t+1}\n-------------------------------")
    train_loop(train_data_tensor, model, loss_fn, optimizer)
    
print("Done")

Can someone help me rectify errors in code?

SolveForum.com may not be responsible for the answers or solutions given to any question asked by the users. All Answers or responses are user generated answers and we do not have proof of its validity or correctness. Please vote for the answer that helped you in order to help others find out which is the most helpful answer. Questions labeled as solved may be solved or may not be solved depending on the type of question and the date posted for some posts may be scheduled to be deleted periodically. Do not hesitate to share your thoughts here to help others.

MediaStore.Audio.Media.EXTERNAL_CONTENT_URI is deprecated for api 29+?

efesydss Asks: MediaStore.Audio.Media.EXTERNAL_CONTENT_URI is deprecated for api 29+?
I am developing an offline music player application. For this, I get the data of the music with MediaStore.Audio.Media.DATA and a similar structure and list them on the recycler view. The code works fine up to api level 29, but not working api 29 and later. On the Google MediStore Documents page, it is written that the MediaStore library is deprecated from api 29+ and later. So what should I do for a system running after 29+? I couldn't find a source or sample code for it. If I can solve this, I will also share the Github link of my project. I am a Junior developer, I will be very happy if anyone can help or suggest resources.The method I wrote that works without api 29 and before;

Code:
public void loadData() {
    ContentResolver contentResolver = requireContext().getContentResolver();

    Uri uri = MediaStore.Audio.Media.EXTERNAL_CONTENT_URI;
    String selection = MediaStore.Audio.Media.IS_MUSIC + "!= 0";
    String sortOrder = TITLE + " ASC";
    Cursor cursor = contentResolver.query(uri, null, selection, null, sortOrder);

    if (cursor != null && cursor.getCount() > 0) {
        model = new ArrayList<>();
        while (cursor.moveToNext()) {

            @SuppressLint("Range") String data = cursor.getString(cursor.getColumnIndex(MediaStore.Audio.Media.DATA));@SuppressLint("Range") String title = cursor.getString(cursor.getColumnIndex(MediaStore.MediaColumns.TITLE));
            @SuppressLint("Range") String album = cursor.getString(cursor.getColumnIndex(MediaStore.Audio.Media.ALBUM));
            @SuppressLint("Range") String artist = cursor.getString(cursor.getColumnIndex(MediaStore.Audio.Media.ARTIST));
            @SuppressLint("Range") String duration = cursor.getString(cursor.getColumnIndex(MediaStore.Audio.Media.DURATION));
            @SuppressLint("Range") String displayName = cursor.getString(cursor.getColumnIndex(MediaStore.Audio.Media.DISPLAY_NAME));
            @SuppressLint("Range") String volume = cursor.getString(cursor.getColumnIndex(MediaStore.Audio.Media.VOLUME_NAME));
            @SuppressLint("Range") String bucketName = cursor.getString(cursor.getColumnIndex(MediaStore.Audio.Media.BUCKET_DISPLAY_NAME));
            // Save to audioList
            int calculatedDuration = (Integer.valueOf(duration) / 60);
            model.add(new MusicModel(data, title, album, artist));
            Log.e("test",
                    "\n data = " + data +
                            "\n" + "title = " + title +
                            "\n" + "album = " + album +
                            "\n" + "artist = " + artist +
                            "\n" + "duration = " + calculatedDuration +
                            "\n" + "display name = " + displayName +
                            "\n" + "volume name = " + volume +
                            "\n" + "bucket name = " + bucketName);


        }
    }
    cursor.close();
    ArtistAdapter adapter = new ArtistAdapter(requireContext(), model);
    binding.rvArtist.setAdapter(adapter);
    binding.rvArtist.setHasFixedSize(true);
}

so this code works perfectly until api 29. but I don't know of an alternative that I can use on 29+ and later. What should I do?

SolveForum.com may not be responsible for the answers or solutions given to any question asked by the users. All Answers or responses are user generated answers and we do not have proof of its validity or correctness. Please vote for the answer that helped you in order to help others find out which is the most helpful answer. Questions labeled as solved may be solved or may not be solved depending on the type of question and the date posted for some posts may be scheduled to be deleted periodically. Do not hesitate to share your thoughts here to help others.

i am trying to make a website should get data from database and add new data to database

Hüseyin Arda Gültekin Asks: i am trying to make a website should get data from database and add new data to database
i already learned html and css and the half of javascript, with these languages i coded a cool mainpage,working and outputing text editor and not working login-signup page. after all these information i need a database editor tech to get the text that users wrote then add it to database and get data from database then write it to the page will text show theirselves.picture of mainpage

the picture of text editor and output

SolveForum.com may not be responsible for the answers or solutions given to any question asked by the users. All Answers or responses are user generated answers and we do not have proof of its validity or correctness. Please vote for the answer that helped you in order to help others find out which is the most helpful answer. Questions labeled as solved may be solved or may not be solved depending on the type of question and the date posted for some posts may be scheduled to be deleted periodically. Do not hesitate to share your thoughts here to help others.

Latest posts

Top