[Herddb-dev] R: HerdDB collections in production

[Alessandro Luccaroni - Diennea]

Some update on the numbers: I’ve run the same activity in production on MapDB v3.0.7 (with memory mapped files) and HerdDB v0.11.9 and we have a ~13x speedup (6200 sec vs 450 sec)

All this with slightly lower CPU usage, same memory footprint (with the advantage that all the memory is inside the JVM heap, so we don’t see OOM error from the OS anymore) and a bit more GC load (this is expected since we don’t use direct memory anymore).

Ale

Da: herddb-dev-bounces at lists.herddb.org <herddb-dev-bounces at lists.herddb.org> Per conto di Enrico Olivelli
Inviato: giovedì 22 agosto 2019 18:39
A: Herddb developers <herddb-dev at lists.herddb.org>
Oggetto: Re: [Herddb-dev] HerdDB collections in production


We first tried to switch with herd v0.11.0 but we encountered some problems on services with lots of data being continuisly load and unloaded (mainly related to filesystem usage)…you can see an history of the fixes needed in the release histo fo the v0.11 branch, thanks Enrico and Diego!

I don’t have hard numbers on hand (I need to switch back to MapDB and perform some comparisons, I will update you tomorrow on that), but the performance are at least in line (I think they are better) with the added benefit of the controlled memory usage and a BIG reduction in storage IOPS (Herd data storage versus the memory mapped files of MapDB).
PS: @Amit I’ve found this benchmark used in past by MapDB (https://github.com/jankotek/mapdb-benchmarks), maybe it would be good to integrate some bench regarding Collections before starting advertise this feature of herd
Sure
We can wait for 0.12.0 release


Enrico

________________________________

CONFIDENTIALITY & PRIVACY NOTICE
This e-mail (including any attachments) is strictly confidential and may also contain privileged information. If you are not the intended recipient you are not authorised to read, print, save, process or disclose this message. If you have received this message by mistake, please inform the sender immediately and destroy this e-mail, its attachments and any copies. Any use, distribution, reproduction or disclosure by any person other than the intended recipient is strictly prohibited and the person responsible may incur in penalties.
The use of this e-mail is only for professional purposes; there is no guarantee that the correspondence towards this e-mail will be read only by the recipient, because, under certain circumstances, there may be a need to access this email by third subjects belonging to the Company.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.herddb.org/pipermail/herddb-dev/attachments/20190823/c087ab4d/attachment-0001.html>