Can I minimize a mysterious function by running a gradient decent on her neural net approximations? [closed]

V

Vladimir Zolotov

Guest
Vladimir Zolotov Asks: Can I minimize a mysterious function by running a gradient decent on her neural net approximations? [closed]
A cross post from on AI StackExchange, and MathOverflow

So I have this function let call her $F:[0,1]^n \rightarrow \mathbb{R}$ and say $10 \le n \le 100$. I want to find some $x_0 \in [0,1]^n$ such that $F(x_0)$ is as small as possible. I don't think there is any hope of getting the global minimum. I just want a reasonably good $x_0$.

AFAIK the standard approach is to run an (accelerated) gradient decent a bunch of times and take the best result. But in my case values of $F$ are computed algorithmically and I don't have a way to compute gradients for $F$.

So I want to do something like this.

(A) We create a neural network which takes an $n$-dimensional vector as input and returns a real number as result. We want the NN to "predict" values of $F$ but at this point it is untrained.

(B) We take bunch of random points in $[0,1]^n$. We compute values of $F$ at those points. And we train NN using this data.

(C1) Now the neural net provides us with a reasonably smooth function $F_1:[0,1]^n \rightarrow \mathbb{R}$ approximating $F$. We run a gradient decent a bunch of times on $F_1$. We take the final points of those decent and compute $F$ on them to see if we caught any small values. Then we take whole paths of those gradient decent, compute $F$ on them and use this as data to retrain our neural net.

(C2) The retrained neural net provides us with a new function $F_2$ and we repeat the previous step

(C3) ...

Does this approach have a name? Is it used somewhere? Should I indeed use neural nets or there are better ways of constructing smooth approximations for my needs?

SolveForum.com may not be responsible for the answers or solutions given to any question asked by the users. All Answers or responses are user generated answers and we do not have proof of its validity or correctness. Please vote for the answer that helped you in order to help others find out which is the most helpful answer. Questions labeled as solved may be solved or may not be solved depending on the type of question and the date posted for some posts may be scheduled to be deleted periodically. Do not hesitate to share your thoughts here to help others.
 

Unreplied Threads

What is the exact row-reduced echelon from R of A?

SherringFord69 Asks: What is the exact row-reduced echelon from R of A?
Suppose A is a 4 by 5 matrix and s = (4,5,2,1,0) is the only special solution for Ax = 0. What is the exact row-reduced echelon from R of A? Can Ax = b be solved for all b?

SolveForum.com may not be responsible for the answers or solutions given to any question asked by the users. All Answers or responses are user generated answers and we do not have proof of its validity or correctness. Please vote for the answer that helped you in order to help others find out which is the most helpful answer. Questions labeled as solved may be solved or may not be solved depending on the type of question and the date posted for some posts may be scheduled to be deleted periodically. Do not hesitate to share your thoughts here to help others.

Maximum matching for bipartite graph with category restriction.

picker Asks: Maximum matching for bipartite graph with category restriction.
The general maximum matching on bipartite graphs is extracted as follows: We have a bipartite graph G, which contains two sets of vertices without any connections within the two sets and several connections between the two sets. We want to find the maximum matching (number of connections) where there are no two selected edges connected to the same vertex.

Consider the following restriction. If the vertices on the left-hand side ${X}$ have one or several specific label, e.g., $x_1 = 1, x_2 = 2, x_3 = 2, x_4 = {1 and 3} ...$

If we want to achieve that, finally, the match has uniform label distribution. How can we design the algorithm to calculate the maximum matching number?

For example, we have two kinds of labels, 1 and 2. The final matching should have $\frac{k}{2}$ matchings with $label = 1$ and $\frac{k}{2}$ matchings with $label = 2$ if the final matching number is $k$.

SolveForum.com may not be responsible for the answers or solutions given to any question asked by the users. All Answers or responses are user generated answers and we do not have proof of its validity or correctness. Please vote for the answer that helped you in order to help others find out which is the most helpful answer. Questions labeled as solved may be solved or may not be solved depending on the type of question and the date posted for some posts may be scheduled to be deleted periodically. Do not hesitate to share your thoughts here to help others.

Is the converse of Vertically opposite angles true?

Apoorva Shukla Asks: Is the converse of Vertically opposite angles true?
When two straight lines intersect the vertically opposite angles are equal.

But, can we say that, if the vertically opposite angles of two lines are equal then the lines are straight?



The angles marked equal in orange, come after comparing the two congruent triangles, in the attached image.

SolveForum.com may not be responsible for the answers or solutions given to any question asked by the users. All Answers or responses are user generated answers and we do not have proof of its validity or correctness. Please vote for the answer that helped you in order to help others find out which is the most helpful answer. Questions labeled as solved may be solved or may not be solved depending on the type of question and the date posted for some posts may be scheduled to be deleted periodically. Do not hesitate to share your thoughts here to help others.

Intersection of conic sets

bosky2001 Asks: Intersection of conic sets
Question Prompt

How do I find the Intersection of C and C'?

and In addition to this, if I am given a set for C (C definition) , How do i construct the C' of it?

SolveForum.com may not be responsible for the answers or solutions given to any question asked by the users. All Answers or responses are user generated answers and we do not have proof of its validity or correctness. Please vote for the answer that helped you in order to help others find out which is the most helpful answer. Questions labeled as solved may be solved or may not be solved depending on the type of question and the date posted for some posts may be scheduled to be deleted periodically. Do not hesitate to share your thoughts here to help others.

How will you solve this integral? [closed]

Dhruv Mohta Asks: How will you solve this integral? [closed]
the question

we have to solve using only rectangular coordinates, nothing else

SolveForum.com may not be responsible for the answers or solutions given to any question asked by the users. All Answers or responses are user generated answers and we do not have proof of its validity or correctness. Please vote for the answer that helped you in order to help others find out which is the most helpful answer. Questions labeled as solved may be solved or may not be solved depending on the type of question and the date posted for some posts may be scheduled to be deleted periodically. Do not hesitate to share your thoughts here to help others.

Could a Legendre transform be equivalent to evaluation in a retarded time?

JM3 Asks: Could a Legendre transform be equivalent to evaluation in a retarded time?
We have the following function:

U = U(r(t), p(t)) = (α/r)·sqrt(1 - p^2/c^2)

being α and c positive constants, t a real variable 0 or positive, and p = dr/dt.

I would like to show that if we define

t'= t – r(t)/c

it is true that

U' = U - p·(∂U/∂p)

where the prime ' stands for evaluated at t'

SolveForum.com may not be responsible for the answers or solutions given to any question asked by the users. All Answers or responses are user generated answers and we do not have proof of its validity or correctness. Please vote for the answer that helped you in order to help others find out which is the most helpful answer. Questions labeled as solved may be solved or may not be solved depending on the type of question and the date posted for some posts may be scheduled to be deleted periodically. Do not hesitate to share your thoughts here to help others.

What is the sign of Ax where A is negative definite matrix and x > 0?

  • Tomoki Matsumoto
  • Mathematics
  • Replies: 0
Tomoki Matsumoto Asks: What is the sign of Ax where A is negative definite matrix and x > 0?
Let A be a negative definite matrix and x is a vector whose elements are positive all, that is x > 0. Then, is the sign of all elements of the vector Ax negative?

Best regards,

SolveForum.com may not be responsible for the answers or solutions given to any question asked by the users. All Answers or responses are user generated answers and we do not have proof of its validity or correctness. Please vote for the answer that helped you in order to help others find out which is the most helpful answer. Questions labeled as solved may be solved or may not be solved depending on the type of question and the date posted for some posts may be scheduled to be deleted periodically. Do not hesitate to share your thoughts here to help others.