Employing modern C++ for high performance delta-coding compression
C++ is the ideal language for tasks such as compressing time-series data with the technique of delta encoding, because of how well it is suited for performance maximizing with the benefits of composability and other software engineering quality benefits.
This presentation will showcase several things:
- The application of C++ "tricks" to achieve something similar to "reflection" (introspection) so that users can describe their data layouts and tie them to compression parameters and options.
- A good explanation of Delta compression, including prior work by the inventor of the Generic Programming Paradigm, Alexander Stepanov, who, among collaborators, used this technique.
- It is possible for some applications, such as dissemination of financial exchange market data, that the compression is fast enough and reduce enough data to be disseminated that pays for itself in reduced latencies!
- Outlining columnar databases relying on delta compression
- Several counter-intuitive principles about how to micro benchmark very low latency code, typically written in C++.
We will rely on an open source library developed by me to accomplish the topics highlighted.
High Performance and Generic Programming C++ Software Engineer, with experience in financial technologies and other fields. Presenter several times at users groups and C++ conferences