Benchmarks¶
asammdf relies heavily on dict objects. Starting with Python 3.6 the dict objects are more compact and ordered (implementation detail); asammdf uses takes advantage of those changes so for best performance it is advised to use Python >= 3.6.
Test setup¶
The benchmarks were done using two test files (available here https://github.com/danielhrisca/asammdf/issues/14) (for mdf version 3 and 4) of around 170MB. The files contain 183 data groups and a total of 36424 channels.
asamdf 4.0.0.dev was compared against mdfreader 2.7.8. mdfreader seems to be the most used Python package to handle MDF files, and it also supports both version 3 and 4 of the standard.
The three benchmark cathegories are file open, file save and extracting the data for all channels inside the file(36424 calls). For each cathegory two aspect were noted: elapsed time and peak RAM usage.
Dependencies¶
You will need the following packages to be able to run the benchmark script
- psutil
- mdfreader
Usage¶
Extract the test files from the archive, or provide a folder that contains the files “test.mdf” and “test.mf4”. Run the module bench.py ( see –help option for available options )
x64 Python results¶
Benchmark environment
- 3.6.2 (v3.6.2:5fd33b5, Jul 8 2017, 04:57:36) [MSC v.1900 64 bit (AMD64)]
- Windows-10-10.0.17134-SP0
- Intel64 Family 6 Model 69 Stepping 1, GenuineIntel
- 16GB installed RAM
Notations used in the results
- full = asammdf MDF object created with memory=full (everything loaded into RAM)
- low = asammdf MDF object created with memory=low (raw channel data not loaded into RAM, but metadata loaded to RAM)
- minimum = asammdf MDF object created with memory=full (lowest possible RAM usage)
- compress = mdfreader mdf object created with compression=blosc
- noDataLoading = mdfreader mdf object read with noDataLoading=True
Files used for benchmark:
- mdf version 3.10
- 167 MB file size
- 183 groups
- 36424 channels
- mdf version 4.00
- 183 MB file size
- 183 groups
- 36424 channels
Open file | Time [ms] | RAM [MB] |
---|---|---|
asammdf 4.0.0.dev full mdfv3 | 1466 | 337 |
asammdf 4.0.0.dev low mdfv3 | 1372 | 184 |
asammdf 4.0.0.dev minimum mdfv3 | 420 | 70 |
mdfreader 2.7.8 mdfv3 | 2794 | 430 |
mdfreader 2.7.8 compress mdfv3 | 4323 | 129 |
mdfreader 2.7.8 noDataLoading mdfv3 | 1187 | 176 |
asammdf 4.0.0.dev full mdfv4 | 1786 | 312 |
asammdf 4.0.0.dev low mdfv4 | 1637 | 147 |
asammdf 4.0.0.dev minimum mdfv4 | 1099 | 71 |
mdfreader 2.7.8 mdfv4 | 6706 | 441 |
mdfreader 2.7.8 compress mdfv4 | 8542 | 141 |
mdfreader 2.7.8 noDataLoading mdfv4 | 4539 | 182 |
Save file | Time [ms] | RAM [MB] |
---|---|---|
asammdf 4.0.0.dev full mdfv3 | 894 | 343 |
asammdf 4.0.0.dev low mdfv3 | 866 | 190 |
asammdf 4.0.0.dev minimum mdfv3 | 3135 | 78 |
mdfreader 2.7.8 mdfv3 | 7733 | 459 |
mdfreader 2.7.8 noDataLoading mdfv3 | 9353 | 520 |
mdfreader 2.7.8 compress mdfv3 | 7827 | 428 |
asammdf 4.0.0.dev full mdfv4 | 982 | 316 |
asammdf 4.0.0.dev low mdfv4 | 974 | 157 |
asammdf 4.0.0.dev minimum mdfv4 | 3600 | 80 |
mdfreader 2.7.8 mdfv4 | 4669 | 459 |
mdfreader 2.7.8 noDataLoading mdfv4 | 6612 | 478 |
mdfreader 2.7.8 compress mdfv4 | 4525 | 418 |
Get all channels (36424 calls) | Time [ms] | RAM [MB] |
---|---|---|
asammdf 4.0.0.dev full mdfv3 | 1605 | 346 |
asammdf 4.0.0.dev low mdfv3 | 7224 | 199 |
asammdf 4.0.0.dev minimum mdfv3 | 9965 | 87 |
mdfreader 2.7.8 mdfv3 | 102 | 430 |
mdfreader 2.7.8 nodata mdfv3 | 16696 | 211 |
mdfreader 2.7.8 compress mdfv3 | 622 | 129 |
asammdf 4.0.0.dev full mdfv4 | 1685 | 316 |
asammdf 4.0.0.dev low mdfv4 | 12592 | 157 |
asammdf 4.0.0.dev minimum mdfv4 | 16428 | 84 |
mdfreader 2.7.8 mdfv4 | 93 | 441 |
mdfreader 2.7.8 compress mdfv4 | 624 | 141 |
mdfreader 2.7.8 nodata mdfv4 | 27146 | 206 |
Convert file | Time [ms] | RAM [MB] |
---|---|---|
asammdf 4.0.0.dev full v3 to v4 | 5677 | 680 |
asammdf 4.0.0.dev low v3 to v4 | 5737 | 352 |
asammdf 4.0.0.dev minimum v3 to v4 | 9341 | 118 |
asammdf 4.0.0.dev full v4 to v3 | 5095 | 610 |
asammdf 4.0.0.dev low v4 to v3 | 5328 | 263 |
asammdf 4.0.0.dev minimum v4 to v3 | 9983 | 115 |
Merge 3 files | Time [ms] | RAM [MB] |
---|---|---|
asammdf 4.0.0.dev full v3 | 17059 | 1641 |
asammdf 4.0.0.dev low v3 | 16730 | 622 |
asammdf 4.0.0.dev minimum v3 | 25156 | 166 |
mdfreader 2.7.8 v3 | 24608 | 1335 |
mdfreader 2.7.8 compress v3 | 30669 | 1347 |
mdfreader 2.7.8 nodata v3 | 24093 | 1456 |
asammdf 4.0.0.dev full v4 | 17949 | 1513 |
asammdf 4.0.0.dev low v4 | 17592 | 461 |
asammdf 4.0.0.dev minimum v4 | 36417 | 166 |
mdfreader 2.7.8 v4 | 36287 | 1326 |
mdfreader 2.7.8 nodata v4 | 35904 | 1361 |
mdfreader 2.7.8 compress v4 | 42410 | 1336 |
Graphical results¶
(Source code, png, hires.png, pdf)
(Source code, png, hires.png, pdf)
(Source code, png, hires.png, pdf)
(Source code, png, hires.png, pdf)
(Source code, png, hires.png, pdf)
(Source code, png, hires.png, pdf)
(Source code, png, hires.png, pdf)
(Source code, png, hires.png, pdf)
(Source code, png, hires.png, pdf)
(Source code, png, hires.png, pdf)
x86 Python results¶
Benchmark environment
- 3.6.3 (v3.6.3:2c5fed8, Oct 3 2017, 17:26:49) [MSC v.1900 32 bit (Intel)]
- Windows-10-10.0.17134-SP0
- Intel64 Family 6 Model 69 Stepping 1, GenuineIntel
- 16GB installed RAM
Notations used in the results
- full = asammdf MDF object created with memory=full (everything loaded into RAM)
- low = asammdf MDF object created with memory=low (raw channel data not loaded into RAM, but metadata loaded to RAM)
- minimum = asammdf MDF object created with memory=full (lowest possible RAM usage)
- compress = mdfreader mdf object created with compression=blosc
- noDataLoading = mdfreader mdf object read with noDataLoading=True
Files used for benchmark:
- mdf version 3.10
- 167 MB file size
- 183 groups
- 36424 channels
- mdf version 4.00
- 183 MB file size
- 183 groups
- 36424 channels
Open file | Time [ms] | RAM [MB] |
---|---|---|
asammdf 4.0.0.dev full mdfv3 | 1792 | 272 |
asammdf 4.0.0.dev low mdfv3 | 1650 | 119 |
asammdf 4.0.0.dev minimum mdfv3 | 556 | 49 |
mdfreader 2.7.8 mdfv3 | 3526 | 394 |
mdfreader 2.7.8 compress mdfv3 | 5373 | 96 |
mdfreader 2.7.8 noDataLoading mdfv3 | 1427 | 108 |
asammdf 4.0.0.dev full mdfv4 | 2181 | 259 |
asammdf 4.0.0.dev low mdfv4 | 2045 | 94 |
asammdf 4.0.0.dev minimum mdfv4 | 1396 | 49 |
mdfreader 2.7.8 mdfv4 | 8189 | 399 |
mdfreader 2.7.8 compress mdfv4 | 10217 | 100 |
mdfreader 2.7.8 noDataLoading mdfv4 | 5227 | 110 |
Save file | Time [ms] | RAM [MB] |
---|---|---|
asammdf 4.0.0.dev full mdfv3 | 1150 | 278 |
asammdf 4.0.0.dev low mdfv3 | 1303 | 125 |
asammdf 4.0.0.dev minimum mdfv3 | 3978 | 54 |
mdfreader 2.7.8 mdfv3 | 9154 | 414 |
mdfreader 2.7.8 noDataLoading mdfv3 | 10710 | 450 |
mdfreader 2.7.8 compress mdfv3 | 8789 | 388 |
asammdf 4.0.0.dev full mdfv4 | 1262 | 262 |
asammdf 4.0.0.dev low mdfv4 | 1382 | 103 |
asammdf 4.0.0.dev minimum mdfv4 | 4015 | 58 |
mdfreader 2.7.8 mdfv4 | 5285 | 418 |
mdfreader 2.7.8 noDataLoading mdfv4 | 8016 | 437 |
mdfreader 2.7.8 compress mdfv4 | 5027 | 382 |
Get all channels (36424 calls) | Time [ms] | RAM [MB] |
---|---|---|
asammdf 4.0.0.dev full mdfv3 | 2052 | 280 |
asammdf 4.0.0.dev low mdfv3 | 21601 | 131 |
asammdf 4.0.0.dev minimum mdfv3 | 24989 | 63 |
mdfreader 2.7.8 mdfv3 | 125 | 394 |
mdfreader 2.7.8 nodata mdfv3 | 33777 | 132 |
mdfreader 2.7.8 compress mdfv3 | 676 | 96 |
asammdf 4.0.0.dev full mdfv4 | 2177 | 263 |
asammdf 4.0.0.dev low mdfv4 | 25377 | 103 |
asammdf 4.0.0.dev minimum mdfv4 | 29231 | 58 |
mdfreader 2.7.8 mdfv4 | 116 | 399 |
mdfreader 2.7.8 compress mdfv4 | 666 | 105 |
mdfreader 2.7.8 nodata mdfv4 | 42202 | 125 |
Convert file | Time [ms] | RAM [MB] |
---|---|---|
asammdf 4.0.0.dev full v3 to v4 | 6679 | 552 |
asammdf 4.0.0.dev low v3 to v4 | 6897 | 221 |
asammdf 4.0.0.dev minimum v3 to v4 | 11120 | 87 |
asammdf 4.0.0.dev full v4 to v3 | 6004 | 511 |
asammdf 4.0.0.dev low v4 to v3 | 6294 | 161 |
asammdf 4.0.0.dev minimum v4 to v3 | 12459 | 79 |
Merge 3 files | Time [ms] | RAM [MB] |
---|---|---|
asammdf 4.0.0.dev full v3 | 20648 | 1411 |
asammdf 4.0.0.dev low v3 | 20279 | 391 |
asammdf 4.0.0.dev minimum v3 | 30807 | 119 |
mdfreader 2.7.8 v3 | 27054 | 1274 |
mdfreader 2.7.8 compress v3 | 32342 | 1288 |
mdfreader 2.7.8 nodata v3 | 26471 | 1343 |
asammdf 4.0.0.dev full v4 | 21532 | 1335 |
asammdf 4.0.0.dev low v4 | 21272 | 280 |
asammdf 4.0.0.dev minimum v4 | 45546 | 111 |
mdfreader 2.7.8 v4 | 40785 | 1260 |
mdfreader 2.7.8 nodata v4 | 40467 | 1284 |
Graphical results¶
(Source code, png, hires.png, pdf)
(Source code, png, hires.png, pdf)
(Source code, png, hires.png, pdf)
(Source code, png, hires.png, pdf)
(Source code, png, hires.png, pdf)
(Source code, png, hires.png, pdf)
(Source code, png, hires.png, pdf)
(Source code, png, hires.png, pdf)
(Source code, png, hires.png, pdf)
(Source code, png, hires.png, pdf)