The holidays are around the corner and so I have a little crat...eh..._gift_ for you my lil' crabbies.π€ π¦
So here's the plan:
Until December 23rd, I'll post some performance progress I've made over the last months in this thread until we can finally unwrap() the gift together, Ok()?
Mhh...I can already smell the oxidized metal...π€€
Commit e96349f
Yeah...there were times, were I _really_ struggled...
Commit 9d1d7ae
Hell yeah! More than twice as fast! And we are now talking milliseconds, not seconds anymore.
But this is Rust - can we get even faster?
Wait until tomorrow...π
nighty night π
Commit 150b7c8
...and we keep improving!
Now down to ~700ms. That's another 12% faster!
Hm...might we even be able to...
Unfortunately, I can't release this as planned.π₯
I'm so sorry, but I found a very weird (concurrency!) bug and I don't want to ship it with this bug in it. Real life of a developer, I guess...π
(I found this bug writing and executing a specific doc test...What!?)
I'm so disappointed right now.π
I'll look into it after xmas and keep you updated.
I wish you some happy holidays! π
Yay, I fixed the bug!π
It wasn't a concurrency bug in the traditional sense (it's #Rust after all Β―\_(γ)_/Β―), but rather a code path that was sometimes executed and sometimes not, depending on whether the other thread had a chance to run or not.
New plan: release this within 2021 (as alpha)!π π€
Commit 487efd1
Where did we leave off? Oh, right...might we even be able to...
...CROSS THE 500ms mark!?π±
π
Next toot will be the official release. Stay tuned.π
Introducing csv-diff - the fastest CSV-diffing library in the world - written in #Rust!π π
Highlights:
- compare two CSVs with 1,000,000 rows x 9 columns in under 500ms
- thread-pool agnostic π§΅π§Ά
Crates.io:
https://crates.io/crates/csv-diff
GitLab:
https://gitlab.com/janriemer/csv-diff
This crate has been made possible by the awesome Rust community! Thank you!π€ β€οΈ
#CSV #CSVDiff #Performance #Diff #Compare #Fast #CrateTip #Crate #RustLang #OpenSource
Commit 48e91bc
Oops, turns out you can now diff 1 million rows of #CSV in only 400ms.π
All I had to do was to use ahash::AHashMap instead of std HashMap. Β―\_(γ)_/Β―
#Rust #CSV #CSVDiff #Compare #Diff #Performance #AHash