By default when the MDF object is created all data is loaded into RAM (memory=’full’). This will give you the best performance from asammdf.
However if you reach the physical memory limit asammdf gives you two options:
- memory=’low’ : only the metadata is loaded into RAM, the raw channel data is loaded when needed
- memory=’minimum’ : only minimal data is loaded into RAM.
Advantages
Disadvantages
Use case
Advantages
Disadvantages
Note
it is advised to use the MDF context manager in this case
Use case
Note
See benchmarks for the effects of using the flag
Advantages
Disadvantages
Note
See benchmarks for the effects of using the flag
When the MDF is created with the option “full” all the samples are loaded into RAM and are processed as a signle block. For large files this can lead to MemoryError exceptions (for example trying to merge several GB sized files).
asammdf optimizes memory usage for options “low” and “minimum” by processing samples in fragments. The read fragment size was tuned based on experimental measurements and should give a good compromise between execution time and memory usage.
You can further tune the read fragment size using the configure method, to favor execution speed (using larger fragment sizes) or memory usage (using lower fragment sizes).
The MDF methods (cut, filter, select) are optimized and should be used instead of calling get for several channels. For “low” and “minimum” options the time savings can be dramatic.