Using a for-loop for a reduced set (24)` of smaller datafiles took 1 hour 10 min, while using the Pool with 7 processors took 1 hour 28 min. # METHOD WITH FOR-LOOP - TAKES APPROX 1 HR 10 MINĪrgs2 = ((('CaseD%.3d.sim' % casenumber), arcs, 1) for casenumber in range(1,25)) Results = pool.map_async(get_results, args) # List of input needed for the get_results function:Īrgs = ((('CaseD%.3d.sim' % casenumber), arcs, 1) for casenumber in range(1,25)) # METHOD WITH POOL - TAKES APPROX 1 HR 28 MIN # Also does many other operations for extraction of results # Collects results from external program:Ĭ = model.LinkedStatistics(, 1, objectExtra=OrcFxAPI.oeEndB).TimeSeriesStatistics('Ezy-Angle').Meanĭ = model.LinkedStatistics(, 1, objectExtra=OrcFxAPI.oeEndB).Query('Ezy-Angle', 'Ezy-Angle').ValueAtMaxĮ = model.LinkedStatistics(, 1, objectExtra=OrcFxAPI.oeEndB).Query('Ezy-Angle', 'Ezy-Angle').ValueAtMin Import OrcFxAPI # Package connected to external program Using Multiprocessing.Pool().map_async to pool my tasks. Due to this, extracting data from a single file can be time-consuming. The files are of a form requiring to open the program which created them (OrcaFlex) to extract data. The extraction of data from a file is completely independent of extraction of data from other files. I need to extract data from 1300 files and would therefore like Python to start extracting data from a new file once one extraction is complete. Use some function in python to extract data from many datafiles simultanously using different processors on my computer.