Benchmarks

asammdf relies heavily on dict objects. Starting with Python 3.6 the dict objects are more compact and ordered (implementation detail); asammdf uses takes advantage of those changes so for best performance it is advised to use Python >= 3.6.

Test setup

The benchmarks were done using two test files (available here https://github.com/danielhrisca/asammdf/issues/14) (for mdf version 3 and 4) of around 170MB. The files contain 183 data groups and a total of 36424 channels.

asamdf 4.0.0.dev was compared against mdfreader 2.7.8. mdfreader seems to be the most used Python package to handle MDF files, and it also supports both version 3 and 4 of the standard.

The three benchmark cathegories are file open, file save and extracting the data for all channels inside the file(36424 calls). For each cathegory two aspect were noted: elapsed time and peak RAM usage.

Dependencies

You will need the following packages to be able to run the benchmark script

  • psutil
  • mdfreader

Usage

Extract the test files from the archive, or provide a folder that contains the files “test.mdf” and “test.mf4”. Run the module bench.py ( see –help option for available options )

x64 Python results

Benchmark environment

  • 3.6.2 (v3.6.2:5fd33b5, Jul 8 2017, 04:57:36) [MSC v.1900 64 bit (AMD64)]
  • Windows-10-10.0.17134-SP0
  • Intel64 Family 6 Model 69 Stepping 1, GenuineIntel
  • 16GB installed RAM

Notations used in the results

  • full = asammdf MDF object created with memory=full (everything loaded into RAM)
  • low = asammdf MDF object created with memory=low (raw channel data not loaded into RAM, but metadata loaded to RAM)
  • minimum = asammdf MDF object created with memory=full (lowest possible RAM usage)
  • compress = mdfreader mdf object created with compression=blosc
  • noDataLoading = mdfreader mdf object read with noDataLoading=True

Files used for benchmark:

  • mdf version 3.10
    • 167 MB file size
    • 183 groups
    • 36424 channels
  • mdf version 4.00
    • 183 MB file size
    • 183 groups
    • 36424 channels
Open file Time [ms] RAM [MB]
asammdf 4.0.0.dev full mdfv3 1466 337
asammdf 4.0.0.dev low mdfv3 1372 184
asammdf 4.0.0.dev minimum mdfv3 420 70
mdfreader 2.7.8 mdfv3 2794 430
mdfreader 2.7.8 compress mdfv3 4323 129
mdfreader 2.7.8 noDataLoading mdfv3 1187 176
asammdf 4.0.0.dev full mdfv4 1786 312
asammdf 4.0.0.dev low mdfv4 1637 147
asammdf 4.0.0.dev minimum mdfv4 1099 71
mdfreader 2.7.8 mdfv4 6706 441
mdfreader 2.7.8 compress mdfv4 8542 141
mdfreader 2.7.8 noDataLoading mdfv4 4539 182
Save file Time [ms] RAM [MB]
asammdf 4.0.0.dev full mdfv3 894 343
asammdf 4.0.0.dev low mdfv3 866 190
asammdf 4.0.0.dev minimum mdfv3 3135 78
mdfreader 2.7.8 mdfv3 7733 459
mdfreader 2.7.8 noDataLoading mdfv3 9353 520
mdfreader 2.7.8 compress mdfv3 7827 428
asammdf 4.0.0.dev full mdfv4 982 316
asammdf 4.0.0.dev low mdfv4 974 157
asammdf 4.0.0.dev minimum mdfv4 3600 80
mdfreader 2.7.8 mdfv4 4669 459
mdfreader 2.7.8 noDataLoading mdfv4 6612 478
mdfreader 2.7.8 compress mdfv4 4525 418
Get all channels (36424 calls) Time [ms] RAM [MB]
asammdf 4.0.0.dev full mdfv3 1605 346
asammdf 4.0.0.dev low mdfv3 7224 199
asammdf 4.0.0.dev minimum mdfv3 9965 87
mdfreader 2.7.8 mdfv3 102 430
mdfreader 2.7.8 nodata mdfv3 16696 211
mdfreader 2.7.8 compress mdfv3 622 129
asammdf 4.0.0.dev full mdfv4 1685 316
asammdf 4.0.0.dev low mdfv4 12592 157
asammdf 4.0.0.dev minimum mdfv4 16428 84
mdfreader 2.7.8 mdfv4 93 441
mdfreader 2.7.8 compress mdfv4 624 141
mdfreader 2.7.8 nodata mdfv4 27146 206
Convert file Time [ms] RAM [MB]
asammdf 4.0.0.dev full v3 to v4 5677 680
asammdf 4.0.0.dev low v3 to v4 5737 352
asammdf 4.0.0.dev minimum v3 to v4 9341 118
asammdf 4.0.0.dev full v4 to v3 5095 610
asammdf 4.0.0.dev low v4 to v3 5328 263
asammdf 4.0.0.dev minimum v4 to v3 9983 115
Merge 3 files Time [ms] RAM [MB]
asammdf 4.0.0.dev full v3 17059 1641
asammdf 4.0.0.dev low v3 16730 622
asammdf 4.0.0.dev minimum v3 25156 166
mdfreader 2.7.8 v3 24608 1335
mdfreader 2.7.8 compress v3 30669 1347
mdfreader 2.7.8 nodata v3 24093 1456
asammdf 4.0.0.dev full v4 17949 1513
asammdf 4.0.0.dev low v4 17592 461
asammdf 4.0.0.dev minimum v4 36417 166
mdfreader 2.7.8 v4 36287 1326
mdfreader 2.7.8 nodata v4 35904 1361
mdfreader 2.7.8 compress v4 42410 1336

x86 Python results

Benchmark environment

  • 3.6.3 (v3.6.3:2c5fed8, Oct 3 2017, 17:26:49) [MSC v.1900 32 bit (Intel)]
  • Windows-10-10.0.17134-SP0
  • Intel64 Family 6 Model 69 Stepping 1, GenuineIntel
  • 16GB installed RAM

Notations used in the results

  • full = asammdf MDF object created with memory=full (everything loaded into RAM)
  • low = asammdf MDF object created with memory=low (raw channel data not loaded into RAM, but metadata loaded to RAM)
  • minimum = asammdf MDF object created with memory=full (lowest possible RAM usage)
  • compress = mdfreader mdf object created with compression=blosc
  • noDataLoading = mdfreader mdf object read with noDataLoading=True

Files used for benchmark:

  • mdf version 3.10
    • 167 MB file size
    • 183 groups
    • 36424 channels
  • mdf version 4.00
    • 183 MB file size
    • 183 groups
    • 36424 channels
Open file Time [ms] RAM [MB]
asammdf 4.0.0.dev full mdfv3 1792 272
asammdf 4.0.0.dev low mdfv3 1650 119
asammdf 4.0.0.dev minimum mdfv3 556 49
mdfreader 2.7.8 mdfv3 3526 394
mdfreader 2.7.8 compress mdfv3 5373 96
mdfreader 2.7.8 noDataLoading mdfv3 1427 108
asammdf 4.0.0.dev full mdfv4 2181 259
asammdf 4.0.0.dev low mdfv4 2045 94
asammdf 4.0.0.dev minimum mdfv4 1396 49
mdfreader 2.7.8 mdfv4 8189 399
mdfreader 2.7.8 compress mdfv4 10217 100
mdfreader 2.7.8 noDataLoading mdfv4 5227 110
Save file Time [ms] RAM [MB]
asammdf 4.0.0.dev full mdfv3 1150 278
asammdf 4.0.0.dev low mdfv3 1303 125
asammdf 4.0.0.dev minimum mdfv3 3978 54
mdfreader 2.7.8 mdfv3 9154 414
mdfreader 2.7.8 noDataLoading mdfv3 10710 450
mdfreader 2.7.8 compress mdfv3 8789 388
asammdf 4.0.0.dev full mdfv4 1262 262
asammdf 4.0.0.dev low mdfv4 1382 103
asammdf 4.0.0.dev minimum mdfv4 4015 58
mdfreader 2.7.8 mdfv4 5285 418
mdfreader 2.7.8 noDataLoading mdfv4 8016 437
mdfreader 2.7.8 compress mdfv4 5027 382
Get all channels (36424 calls) Time [ms] RAM [MB]
asammdf 4.0.0.dev full mdfv3 2052 280
asammdf 4.0.0.dev low mdfv3 21601 131
asammdf 4.0.0.dev minimum mdfv3 24989 63
mdfreader 2.7.8 mdfv3 125 394
mdfreader 2.7.8 nodata mdfv3 33777 132
mdfreader 2.7.8 compress mdfv3 676 96
asammdf 4.0.0.dev full mdfv4 2177 263
asammdf 4.0.0.dev low mdfv4 25377 103
asammdf 4.0.0.dev minimum mdfv4 29231 58
mdfreader 2.7.8 mdfv4 116 399
mdfreader 2.7.8 compress mdfv4 666 105
mdfreader 2.7.8 nodata mdfv4 42202 125
Convert file Time [ms] RAM [MB]
asammdf 4.0.0.dev full v3 to v4 6679 552
asammdf 4.0.0.dev low v3 to v4 6897 221
asammdf 4.0.0.dev minimum v3 to v4 11120 87
asammdf 4.0.0.dev full v4 to v3 6004 511
asammdf 4.0.0.dev low v4 to v3 6294 161
asammdf 4.0.0.dev minimum v4 to v3 12459 79
Merge 3 files Time [ms] RAM [MB]
asammdf 4.0.0.dev full v3 20648 1411
asammdf 4.0.0.dev low v3 20279 391
asammdf 4.0.0.dev minimum v3 30807 119
mdfreader 2.7.8 v3 27054 1274
mdfreader 2.7.8 compress v3 32342 1288
mdfreader 2.7.8 nodata v3 26471 1343
asammdf 4.0.0.dev full v4 21532 1335
asammdf 4.0.0.dev low v4 21272 280
asammdf 4.0.0.dev minimum v4 45546 111
mdfreader 2.7.8 v4 40785 1260
mdfreader 2.7.8 nodata v4 40467 1284