Intro

The benchmarks were done using two test files (for mdf version 3 and 4) of around 170MB. The files contain 183 data groups and a total of 36424 channels.

asamdf 2.3.0 was compared against mdfreader 0.2.5. mdfreader seems to be the most used Python package to handle MDF files, and it also supports both version 3 and 4 of the standard.

The three benchmark cathegories are file open, file save and extracting the data for all channels inside the file(36424 calls). For each cathegory two aspect were noted: elapsed time and peak RAM usage.

Dependencies

You will need the following packages to be able to run the benchmark script

  • psutil
  • mdfreader

x64 Python results

The test environment used for 64 bit tests had:

  • 3.6.2 (v3.6.2:5fd33b5, Jul 8 2017, 04:57:36) [MSC v.1900 64 bit (AMD64)]
  • Windows-10-10.0.14393-SP0
  • Intel64 Family 6 Model 94 Stepping 3, GenuineIntel
  • 16GB installed RAM

Notations used in the results

  • nodata = MDF object created with load_measured_data=False (raw channel data not loaded into RAM)
  • compression = MDF object created with compression=blosc
  • compression bcolz 6 = MDF object created with compression=6
  • noDataLoading = MDF object read with noDataLoading=True

Files used for benchmark: * 183 groups * 36424 channels

Open file Time [ms] RAM [MB]
asammdf 2.3.0 mdfv3 1011 379
asammdf 2.3.0 nodata mdfv3 725 198
mdfreader 0.2.5 mdfv3 2973 537
asammdf 2.3.0 mdfv4 1890 464
asammdf 2.3.0 nodata mdfv4 1542 268
mdfreader 0.2.5 mdfv4 32192 748
Save file Time [ms] RAM [MB]
asammdf 2.3.0 mdfv3 359 379
asammdf 2.3.0 nodata mdfv3 352 205
mdfreader 0.2.5 mdfv3 21777 1997
asammdf 2.3.0 mdfv4 525 471
asammdf 2.3.0 nodata mdfv4 542 280
mdfreader 0.2.5 mdfv4 19591 2795
Get all channels (36424 calls) Time [ms] RAM [MB]
asammdf 2.3.0 mdfv3 589 383
asammdf 2.3.0 nodata mdfv3 8841 209
mdfreader 0.2.5 mdfv3 28 537
asammdf 2.3.0 mdfv4 494 468
asammdf 2.3.0 nodata mdfv4 12330 280
mdfreader 0.2.5 mdfv4 39 748

x86 Python results

The test environment used for 32 bit tests had:

  • Python 3.6.1 (v3.6.1:69c0db5, Mar 21 2017, 17:54:52) [MSC v.1900 32 bit (Intel)]
  • Windows-7-6.1.7601-SP1
  • Intel64 Family 6 Model 94 Stepping 3, GenuineIntel (i7-6820Q)
  • 16GB installed RAM

The notations used in the results have the following meaning:

  • nodata = MDF object created with load_measured_data=False (raw channel data no loaded into RAM)
  • compression = MDF object created with compression=True (raw channel data loaded into RAM and compressed)
  • noconvert = MDF object created with convertAfterRead=False

Raw data

  • 3.6.1 (v3.6.1:69c0db5, Mar 21 2017, 17:54:52) [MSC v.1900 32 bit (Intel)]
  • Windows-10-10.0.14393-SP0
  • Intel64 Family 6 Model 94 Stepping 3, GenuineIntel
  • 16GB installed RAM

Notations used in the results

  • nodata = MDF object created with load_measured_data=False (raw channel data not loaded into RAM)
  • compression = MDF object created with compression=True/blosc
  • compression bcolz 6 = MDF object created with compression=6
  • noDataLoading = MDF object read with noDataLoading=True

Files used for benchmark: * 183 groups * 36424 channels

Open file Time [ms] RAM [MB]
asammdf 2.3.0 mdfv3 1126 294
asammdf 2.3.0 nodata mdfv3 917 123
mdfreader 0.2.5 mdfv3 3743 455
asammdf 2.3.0 mdfv4 2359 348
asammdf 2.3.0 nodata mdfv4 1906 166
mdfreader 0.2.5 mdfv4 43166 577
Save file Time [ms] RAM [MB]
asammdf 2.3.0 mdfv3 420 297
asammdf 2.3.0 nodata mdfv3 445 130
mdfreader 0.2.5 mdfv3 20078 1224
asammdf 2.3.0 mdfv4 711 357
asammdf 2.3.0 nodata mdfv4 738 175
mdfreader 0.2.5 mdfv4 17822 1687
Get all channels (36424 calls) Time [ms] RAM [MB]
asammdf 2.3.0 mdfv3 777 299
asammdf 2.3.0 nodata mdfv3 18662 132
mdfreader 0.2.5 mdfv3 36 455
asammdf 2.3.0 mdfv4 681 354
asammdf 2.3.0 nodata mdfv4 20439 176
mdfreader 0.2.5 mdfv4 51 578