How do can I grab everything from the first occurrence of Grid-ref to the last bit from a file which contains the data in the format below in js

K

Kevin

Guest
Kevin Asks: How do can I grab everything from the first occurrence of Grid-ref to the last bit from a file which contains the data in the format below in js
I am learning to code in javascript and I have been task to extract data from a .pre text file which looks like below and insert the data into a 4 columned DB table which looks like this Xref Yref Date Value Xref is the value 1 from the Grid-ref= 1, 148 line etc Yref is the value 148 from the Grid-ref= 1, 148 line etc date is in the format 1/1/1991 where the first row of values represents Jan 1991 to Dec 1991, the second row Jan 1992 to Dec 1992 etc. value is the corresponding value like this row 3020 2820 3040 2880 1740 1360 980 990 1410 1770 2580 2630 I have written this code to read the file and sort of display it as an array, I now want to grab the bit of the file from the first occurrence of Grid-Ref to the bottom of the over 50,000 rows and then pull/transfer the data into the DB table format described above

I know how to create my own DB and use node js to get, pull, put, delete etc. What i am struggling with is the transform this data into the DB grabbing the correct bits using regex or something similar. You help will be greatly appreciated

Code:
Some sample text is here
.pre = precipitation (mm)
CRU TS 2.1
[Long=-180.00, 180.00] [Lati= -90.00,  90.00] [Grid X,Y= 720, 360]
[Boxes=   67420] [Years=1991-2000] [Multi=    0.1000] [Missing=-999]
Grid-ref=   1, 148
 3020 2820 3040 2880 1740 1360  980  990 1410 1770 2580 2630
 3020 2820 3040 2880 1740 1360  980  990 1410 1770 2580 2630
 3020 2820 3040 2880 1740 1360  980  990 1410 1770 2580 2630
 3020 2820 3040 2880 1740 1360  980  990 1410 1770 2580 2630
 3020 2820 3040 2880 1740 1360  980  990 1410 1770 2580 2630
 3020 2820 3040 2880 1740 1360  980  990 1410 1770 2580 2630
 3020 2820 3040 2880 1740 1360  980  990 1410 1770 2580 2630
 3020 2820 3040 2880 1740 1360  980  990 1410 1770 2580 2630
 3020 2820 3040 2880 1740 1360  980  990 1410 1770 2580 2630
 3020 2820 3040 2880 1740 1360  980  990 1410 1770 2580 2630
Grid-ref=   1, 311
  490  290  280  230  200  250  440  530  460  420  530  450
  490  290  280  230  200  250  440  530  460  420  530  450
  490  290  280  230  200  250  440  530  460  420  530  450```

SolveForum.com may not be responsible for the answers or solutions given to any question asked by the users. All Answers or responses are user generated answers and we do not have proof of its validity or correctness. Please vote for the answer that helped you in order to help others find out which is the most helpful answer. Questions labeled as solved may be solved or may not be solved depending on the type of question and the date posted for some posts may be scheduled to be deleted periodically. Do not hesitate to share your thoughts here to help others.
 

Unreplied Threads

How does backpropagation through accuracy work?

  • Константин Писаный
  • Computer Science
  • Replies: 0
Константин Писаный Asks: How does backpropagation through accuracy work?
I'm using a specific constraint on my predicted logits and adding it to the loss. In a nutshell, this constraint tries to minimize cross-overlap between the channels of my predictions. I'm using accuracy to calculate overlap and I'm curious how does the backpropagation works in this case.

I would appreciate any insights or helpful resources on this matter. Thank you.

SolveForum.com may not be responsible for the answers or solutions given to any question asked by the users. All Answers or responses are user generated answers and we do not have proof of its validity or correctness. Please vote for the answer that helped you in order to help others find out which is the most helpful answer. Questions labeled as solved may be solved or may not be solved depending on the type of question and the date posted for some posts may be scheduled to be deleted periodically. Do not hesitate to share your thoughts here to help others.

How to specify scale_pos_weight value at runtime in Hyperopt?

megjosh Asks: How to specify scale_pos_weight value at runtime in Hyperopt?
I want to use LighgbmClassifier for a binary Classification. for Hyper Parameter tuning I want to use Hyperopt. The Dataset is imbalanced. Using Sklearns class_weight.compute_class_weight as shown below

Code:
        clas_wts_arr = class_weight.compute_class_weight('balanced',np.unique(y_trn),y_trn)   
        self.scale_pos_wt = clas_wts_arr[0] / clas_wts_arr[1]

The following is the space parameter that I am passing to the objective function

Code:
        space = {'objective' : hp.choice('objective', objective_list),
                 'boosting' : hp.choice('boosting', boosting_list),
                 'metric' : hp.choice('metric', metric_list),
                 "max_depth": hp.quniform("max_depth", 1, 15,2),
                 'min_data_in_leaf': hp.quniform('min_data_in_leaf', 1, 256, 1),                     
                 'num_leaves': hp.quniform('num_leaves', 7, 150, 1),
                 'feature_fraction' : hp.quniform('feature_fraction', 0.5, 1, 0.01),
                 'min_gain_to_split' : hp.quniform('min_gain_to_split', 0.1, 5, 0.01),
                 'lambda_l1' : hp.uniform('lambda_l1', 0, 5),
                 'lambda_l2' : hp.uniform('lambda_l2', 0, 5),
                 'feature_pre_filter': False}

My question will the following set scale_pos_weight properly in the space dictionary

Code:
        #set scale pos weight explicitly
        space['scale_pos_weight'] = self.scale_pos_wt

If that is wrong then what would be the correct way to set scale_pos_weight at runtime in the space dictionary that is passed to the Objective fn that is in turn passed to the fmin of Hyperopt.

Thanks for your help and answers.

SolveForum.com may not be responsible for the answers or solutions given to any question asked by the users. All Answers or responses are user generated answers and we do not have proof of its validity or correctness. Please vote for the answer that helped you in order to help others find out which is the most helpful answer. Questions labeled as solved may be solved or may not be solved depending on the type of question and the date posted for some posts may be scheduled to be deleted periodically. Do not hesitate to share your thoughts here to help others.

How to generate spatial scale-free nwtworks?

Criticalola Asks: How to generate spatial scale-free nwtworks?
I want to generate spatial scale-free networks for my project. Are there any python libraries that enable it?

I read about the BA model (https://www.science.org/doi/pdf/10.1126/science.286.5439.509) which seems like the pioneer of scale-free networks but I am not sure it takes the spatial aspect into the account.

Can anyone explain:

  1. What is the difference between a spatial and non-spatial scale-free network?
  2. Python libraries to generate a spatial scale-free network.

SolveForum.com may not be responsible for the answers or solutions given to any question asked by the users. All Answers or responses are user generated answers and we do not have proof of its validity or correctness. Please vote for the answer that helped you in order to help others find out which is the most helpful answer. Questions labeled as solved may be solved or may not be solved depending on the type of question and the date posted for some posts may be scheduled to be deleted periodically. Do not hesitate to share your thoughts here to help others.

Optimal Binary Search Trees Knuth

Varun Asks: Optimal Binary Search Trees Knuth
Knuth, Donald E. (1971), "Optimum binary search trees", Acta Informatica 1 (1): 14–25,doi:10.1007/BF00264289

Please have a look at this paper, specifically page 18 in which he tries to prove his lemma that $R_{0,n-1} \leq R_{0,n} $ here $R$ refers to the minimal optimal node which will be the root of the binary tree containing elements $a_{0} ... a_{n} $ .

I understood the idea of the proof using induction that for some $k \, j_{k}=i_{k} $ . Now the next part of the proof is cutting and replacing,I have understood perfectly till there. What i don't understand is how $F''$ weighted path length is equal to weighted path length of $F'$ for all $a_{n}$. Can anyone please give me a hint or a solution to that.

SolveForum.com may not be responsible for the answers or solutions given to any question asked by the users. All Answers or responses are user generated answers and we do not have proof of its validity or correctness. Please vote for the answer that helped you in order to help others find out which is the most helpful answer. Questions labeled as solved may be solved or may not be solved depending on the type of question and the date posted for some posts may be scheduled to be deleted periodically. Do not hesitate to share your thoughts here to help others.

[Solved] Summing values of different layers based on another

  • user140119
  • Geography
  • Replies: 0
user140119 Asks: Summing values of different layers based on another
I have a layer (called here 'layer_top') made of multiple polygons (in orange in the picture). Each polygon of the 'layer_top' overlaps with polygons of other different layers (called here 1,2,3,...). Each of these layers, within the 'layer_top', has a column with a score value (like in the picture). I would like to get the sum of the scores of all polygons of the layers 1,2,3, ... contained in each polygon of the layer_top. I have searched for a solution in this forum and tried to join attributes by location (including summary), but the result is not what I want. enter image description here enter image description here

SolveForum.com may not be responsible for the answers or solutions given to any question asked by the users. All Answers or responses are user generated answers and we do not have proof of its validity or correctness. Please vote for the answer that helped you in order to help others find out which is the most helpful answer. Questions labeled as solved may be solved or may not be solved depending on the type of question and the date posted for some posts may be scheduled to be deleted periodically. Do not hesitate to share your response here to help other visitors like you. Thank you, solveforum.

[Solved] About getInfo use in earthengine-api (Python)

  • maohaitaocumtb maohaitaocumtb
  • Geography
  • Replies: 0
maohaitaocumtb maohaitaocumtb Asks: About getInfo use in earthengine-api (Python)
When I write the an algorithm in Google Earth Engine using JavaScript by using the Code Editor, it works fast. After thatif I use the Map.addLayer(), the result of algorithm can show in the map quickly.

However, after I translate the algorithm into the version of Python with the earthengine-api and I want to get some intermediate results, I need to use the .getInfo() function. According to the official tutorial, you shouldn't use getInfo() unless you absolutely need to. If you call getInfo() in your code, Earth Engine will open the container and tell you what's inside, but it will block the rest of your code until that's done. So this function will slow down my program.

How can I address this issue in Python?

SolveForum.com may not be responsible for the answers or solutions given to any question asked by the users. All Answers or responses are user generated answers and we do not have proof of its validity or correctness. Please vote for the answer that helped you in order to help others find out which is the most helpful answer. Questions labeled as solved may be solved or may not be solved depending on the type of question and the date posted for some posts may be scheduled to be deleted periodically. Do not hesitate to share your response here to help other visitors like you. Thank you, solveforum.

[Solved] Working with CIRCULARSTRING in OGR's sqlite dialect

  • bugmenot123
  • Geography
  • Replies: 0
bugmenot123 Asks: Working with CIRCULARSTRING in OGR's sqlite dialect
ogrinfo ":memory:" -dialect sqlite -sql "SELECT ST_GeomFromText('Point(1 2)')" gives me a dataset with a geometry column:

Code:
INFO: Open of `:memory:'
      using driver `SQLite' successful.

Layer name: SELECT
Geometry: Unknown (any)
Feature Count: 1
Extent: (1.000000, 2.000000) - (1.000000, 2.000000)
Layer SRS WKT:
(unknown)
Geometry Column = ST_GeomFromText('Point(1 2)')
OGRFeature(SELECT):0
  POINT (1 2)

but this does not work with some curve geometries, e. g. CIRCULARSTRING.

Code:
ogrinfo ":memory:" -dialect sqlite -sql "SELECT ST_GeomFromText('CIRCULARSTRING(1 5, 6 2, 7 3)')"

for example results in:

Code:
INFO: Open of `:memory:'
      using driver `SQLite' successful.

Layer name: SELECT
Geometry: None
Feature Count: 1
Layer SRS WKT:
(unknown)
ST_GeomFromText('CIRCULARSTRING(1 5, 6 2, 7 3)'): String (0.0)
OGRFeature(SELECT):0
  ST_GeomFromText('CIRCULARSTRING(1 5, 6 2, 7 3)') (String) = (null)

How can I work with such geometries when using the sqlite dialect? I would be fine with turning them into linear geometries in the query. Using -nlt CONVERT_TO_LINEAR is not a solution for me. The actual problem I am trying to solve is in third-party software that uses the same logic and there we can only edit the query (QGIS layer filters).

References:


SolveForum.com may not be responsible for the answers or solutions given to any question asked by the users. All Answers or responses are user generated answers and we do not have proof of its validity or correctness. Please vote for the answer that helped you in order to help others find out which is the most helpful answer. Questions labeled as solved may be solved or may not be solved depending on the type of question and the date posted for some posts may be scheduled to be deleted periodically. Do not hesitate to share your response here to help other visitors like you. Thank you, solveforum.