python - Error saving and loading a list of matrices -

-

i have list "data_list", , save in order load in script. first of converted in array, in way:

data_array = np.array(data_list)

then saved it:

np.savez("file", data_array)

then, in script want access "file"; so:

a = np.load("file.npz")  b = a['arr_0']

i used code until 2 weeks ago , worked fine. in these days trying work program, ends error identified in line

b = a['arr_0']

"file" 300 mb file. strangest thing has stopped work. idea can happened?

ps: give information. list contains 180 matrices 511x511. each matrix contains decimal numbers (i tried create 180 matrices of zeros, , error occurs in same way). if reduce number of matrices, script works fine: in particular down 130 matrices ok, while program doesn't work. here report error message

        b = a['arr_0']     file "c:\python27\lib\site-packages\numpy\lib\npyio.py", line 241, in          __getitem__         return format.read_array(value)     file "c:\python27\lib\site-packages\numpy\lib\format.py", line 459, in        read_array         array = numpy.fromstring(data, dtype=dtype, count=count)     memoryerror

memoryerror out of memory condition. explains why happens objects of @ least size - more , bigger arrays, expect, require more memory. max size is, , why seems have changed, harder. can highly specific system, in regard considerations like:

  • how memory (physical ram , swap space) exists , available operating system
  • how virtual memory os gives python
  • how of you're using
  • the implementation of c library, of malloc function, can affect how python uses memory allocated

and possibly quite few other things.

per comments, seems biggest problem here running 32 bit build of python. on windows, 32 bit processes apparently have effective maximum memory address space of around 2gb. tests, list of arrays using might take around quarter of that. fact error comes when reading file in suggests numpy deserialisation relatively memory intensive, don't know enough implementation able why be. in case, seems installing 64 bit build of python best bet.


Comments

Post a Comment

Popular posts from this blog

yii2 - Yii 2 Running a Cron in the basic template -

-
i have seen instructions on running cron in advanced template, cant figure out how on basic template. have following controller in basic controllers folder. <?php namespace app\controllers; use yii\console\controller; /** * cron controller */ class croncontroller extends controller { public function actionindex() { echo "cron service runnning"; } public function actionmail($to) { echo "sending mail " . $to; } } i have navigated root of application , tried these comands yii cron php yii cron im getting unknown comand "cron" i know old question failed find actual answer this. if @ code yii exec file you'll notice requires /common/config/aliases.php file console/config/main.php. can naturally change these or copy on these advanced template. hope solves problem wanting same thing.
Read more

asp.net - 'System.Web.HttpContext' does not contain a definition for 'GetOwinContext' Mystery -

-
i realize question might seem trivial some, it's these types of things find myself fighting quite bit , want make sense of despite seeming losing battle in .net (for me anyway). so, if following: using system.web; ... applicationuser user = system.web.httpcontext.getowincontext().getusermanager<applicationusermanager>().findbyid(system.web.httpcontext.user.identity.getuserid()); that produces error in title , red getowincontext() , error cannot resolve symbol 'getowincontext()' however, if following (remove system.web in front of httpcontext ), works expected (or @ least no errors): using system.web; ... applicationuser user = httpcontext.getowincontext().getusermanager<applicationusermanager>().findbyid(system.web.httpcontext.user.identity.getuserid()); however, if (same line that's working using system.web commented out): //using system.web; ... applicationuser user = httpcontext.getowincontext().getusermana...
Read more

mercurial graft feature, can it copy? -

-
there new feature in mercurial 3, called 'graft' (graft local). job of moving change-set different branch. is there way "copy" change-set branch? i interested in in moving stuff qa branch production branch, still need code reside in qa. want copy. found , have tried few things, maybe it'll become clear more tinkering , reading... does have better way of 'cherry picking' change-sets move cross branch (or trunk)??? graft copies changesets, doesn't move them. hg graft (emphasis mine): this command uses mercurial's merge logic copy individual changes other branches without merging branches in history graph. known 'backporting' or 'cherry-picking' . default, graft copy user, date, , description source changesets.
Read more