Efficiently capture variable number of patterns with Julia regex? -

-

i pulling data out of giant text files sections interested in like

... section:numberofsurvivorspervault subsection:1958 xy:1_1034 xy:2_2334 subsection:1959 xy:1_1334 xy:2_2874 xy:7_12 ... section:meancapsperghoul subsection:1962 xy:1_234 xy:2_121 ....

the sections/subsections randomly scattered throughout text file , have variable numbers of xy pairs. right readall'ing full text , capturing each , adding them dataframe with:

function pushparametricdata(df, full)      m = eachmatch(r"section:(.*)\r\nsubsection:([0-9]*)\r\n((xy:[0-9]*_.*?\r\n)+)"m, full)         r = eachmatch(r"xy:([0-9]+)_(.*?)\r\n"m, m.captures[3])             push!(df, [m.captures[1], int(m.captures[2]), int(r.captures[1]), float(r.captures[2])])         end     end end

this works ok, think allocates @ least twice memory needs due 2 regexes, , @time shows 80% of run gc. there way can done without making intermediate copy? (from can tell it's not possible single regex).

it depends on need validate on rest of text file. instance, if don't need syntax validation, in know sure text file has correct section-subsection-item structure, use regex:

(?:\g|(?:\g|^section:(.*)[\r\n]+)subsection:(\d*)[\r\n]+)xy:(\d*_.*+[\r\n]+)

iterating each xy pair.

example:

for m = eachmatch(r"(?:\g|(?:\g|^section:(.*)[\r\n]+)subsection:(\d*)[\r\n]+)xy:(\d*_.*+[\r\n]+)"m, full)     if m.captures[2] != nothing         sub = m.captures[2]         if m.captures[1] != nothing             sec = m.captures[1]         end     end     item = m.captures[3]      print("section: ", sec, " -- subsection: ", sub, " -- item: ", item) end

*please forgive me, first time try coding in julia.

prints:

section: numberofsurvivorspervault -- subsection: 1958 -- item: 1_1034 section: numberofsurvivorspervault -- subsection: 1958 -- item: 2_2334 section: numberofsurvivorspervault -- subsection: 1959 -- item: 1_1334 section: numberofsurvivorspervault -- subsection: 1959 -- item: 2_2874 section: numberofsurvivorspervault -- subsection: 1959 -- item: 7_12 section: meancapsperghoul -- subsection: 1962 -- item: 1_234 section: meancapsperghoul -- subsection: 1962 -- item: 2_121

this expression uses \g match @ end of last match. try match in order:

  1. if there previous match, try match xy pair in m.captures[3] anchored end of last match, leaving 1st , 2nd capturing group unset.

  2. if (1) doesn't match, try match both subsection , xy pair in m.captures[2] , m.captures[3], again anchored end of last match, leaving 1st capturing group unset.

  3. try perform full match, section, subsection , xy pair

this example work on subject text, , serves starting point working example depending on actual structure of text files. take account fail if have subsections missing example.


Comments

Post a Comment

Popular posts from this blog

yii2 - Yii 2 Running a Cron in the basic template -

-
i have seen instructions on running cron in advanced template, cant figure out how on basic template. have following controller in basic controllers folder. <?php namespace app\controllers; use yii\console\controller; /** * cron controller */ class croncontroller extends controller { public function actionindex() { echo "cron service runnning"; } public function actionmail($to) { echo "sending mail " . $to; } } i have navigated root of application , tried these comands yii cron php yii cron im getting unknown comand "cron" i know old question failed find actual answer this. if @ code yii exec file you'll notice requires /common/config/aliases.php file console/config/main.php. can naturally change these or copy on these advanced template. hope solves problem wanting same thing.
Read more

asp.net - 'System.Web.HttpContext' does not contain a definition for 'GetOwinContext' Mystery -

-
i realize question might seem trivial some, it's these types of things find myself fighting quite bit , want make sense of despite seeming losing battle in .net (for me anyway). so, if following: using system.web; ... applicationuser user = system.web.httpcontext.getowincontext().getusermanager<applicationusermanager>().findbyid(system.web.httpcontext.user.identity.getuserid()); that produces error in title , red getowincontext() , error cannot resolve symbol 'getowincontext()' however, if following (remove system.web in front of httpcontext ), works expected (or @ least no errors): using system.web; ... applicationuser user = httpcontext.getowincontext().getusermanager<applicationusermanager>().findbyid(system.web.httpcontext.user.identity.getuserid()); however, if (same line that's working using system.web commented out): //using system.web; ... applicationuser user = httpcontext.getowincontext().getusermana...
Read more

mercurial graft feature, can it copy? -

-
there new feature in mercurial 3, called 'graft' (graft local). job of moving change-set different branch. is there way "copy" change-set branch? i interested in in moving stuff qa branch production branch, still need code reside in qa. want copy. found , have tried few things, maybe it'll become clear more tinkering , reading... does have better way of 'cherry picking' change-sets move cross branch (or trunk)??? graft copies changesets, doesn't move them. hg graft (emphasis mine): this command uses mercurial's merge logic copy individual changes other branches without merging branches in history graph. known 'backporting' or 'cherry-picking' . default, graft copy user, date, , description source changesets.
Read more