<div dir="ltr">Great news Ale. I will look at integrating the benchmark. I need to catch up to speed with the collections framework Enrico added.</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Aug 23, 2019 at 2:58 AM <<a href="mailto:herddb-dev-request@lists.herddb.org">herddb-dev-request@lists.herddb.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Send herddb-dev mailing list submissions to<br>
<a href="mailto:herddb-dev@lists.herddb.org" target="_blank">herddb-dev@lists.herddb.org</a><br>
<br>
To subscribe or unsubscribe via the World Wide Web, visit<br>
<a href="http://lists.herddb.org/mailman/listinfo/herddb-dev" rel="noreferrer" target="_blank">http://lists.herddb.org/mailman/listinfo/herddb-dev</a><br>
or, via email, send a message with subject or body 'help' to<br>
<a href="mailto:herddb-dev-request@lists.herddb.org" target="_blank">herddb-dev-request@lists.herddb.org</a><br>
<br>
You can reach the person managing the list at<br>
<a href="mailto:herddb-dev-owner@lists.herddb.org" target="_blank">herddb-dev-owner@lists.herddb.org</a><br>
<br>
When replying, please edit your Subject line so it is more specific<br>
than "Re: Contents of herddb-dev digest..."<br>
<br>
<br>
Today's Topics:<br>
<br>
1. HerdDB collections in production (Alessandro Luccaroni - Diennea)<br>
2. Re: HerdDB collections in production (Enrico Olivelli)<br>
3. R: HerdDB collections in production<br>
(Alessandro Luccaroni - Diennea)<br>
<br>
<br>
----------------------------------------------------------------------<br>
<br>
Message: 1<br>
Date: Thu, 22 Aug 2019 15:46:38 +0000<br>
From: Alessandro Luccaroni - Diennea<br>
<<a href="mailto:alessandro.luccaroni@diennea.com" target="_blank">alessandro.luccaroni@diennea.com</a>><br>
To: Herddb developers <<a href="mailto:herddb-dev@lists.herddb.org" target="_blank">herddb-dev@lists.herddb.org</a>><br>
Subject: [Herddb-dev] HerdDB collections in production<br>
Message-ID: <<a href="mailto:dc1d86ab556d4778a4b2b3bf32819659@diennea.com" target="_blank">dc1d86ab556d4778a4b2b3bf32819659@diennea.com</a>><br>
Content-Type: text/plain; charset="us-ascii"<br>
<br>
Hi list,<br>
I'm happy to say that, since the last update of Herd (v0.11.9), we are using HerdDB collections in production system and we are very pleased with the performance and the memory usage of Herd.<br>
<br>
We first tried to switch with herd v0.11.0 but we encountered some problems on services with lots of data being continuisly load and unloaded (mainly related to filesystem usage)...you can see an history of the fixes needed in the release histo fo the v0.11 branch, thanks Enrico and Diego!<br>
<br>
I don't have hard numbers on hand (I need to switch back to MapDB and perform some comparisons, I will update you tomorrow on that), but the performance are at least in line (I think they are better) with the added benefit of the controlled memory usage and a BIG reduction in storage IOPS (Herd data storage versus the memory mapped files of MapDB).<br>
<br>
PS: @Amit I've found this benchmark used in past by MapDB (<a href="https://github.com/jankotek/mapdb-benchmarks" rel="noreferrer" target="_blank">https://github.com/jankotek/mapdb-benchmarks</a>), maybe it would be good to integrate some bench regarding Collections before starting advertise this feature of herd<br>
<br>
Alessandro Luccaroni<br>
Platform Manager @ Diennea - MagNews<br>
Tel.: (+39) 0546 066100 Int. 924 - Mob.: (+39) 393 7273519<br>
Viale G.Marconi 30/14 - 48018 Faenza (RA) - Italy<br>
<br>
<br>
________________________________<br>
<br>
CONFIDENTIALITY & PRIVACY NOTICE<br>
This e-mail (including any attachments) is strictly confidential and may also contain privileged information. If you are not the intended recipient you are not authorised to read, print, save, process or disclose this message. If you have received this message by mistake, please inform the sender immediately and destroy this e-mail, its attachments and any copies. Any use, distribution, reproduction or disclosure by any person other than the intended recipient is strictly prohibited and the person responsible may incur in penalties.<br>
The use of this e-mail is only for professional purposes; there is no guarantee that the correspondence towards this e-mail will be read only by the recipient, because, under certain circumstances, there may be a need to access this email by third subjects belonging to the Company.<br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <<a href="http://lists.herddb.org/pipermail/herddb-dev/attachments/20190822/953888f7/attachment-0001.html" rel="noreferrer" target="_blank">http://lists.herddb.org/pipermail/herddb-dev/attachments/20190822/953888f7/attachment-0001.html</a>><br>
<br>
------------------------------<br>
<br>
Message: 2<br>
Date: Thu, 22 Aug 2019 18:38:49 +0200<br>
From: Enrico Olivelli <<a href="mailto:eolivelli@gmail.com" target="_blank">eolivelli@gmail.com</a>><br>
To: Herddb developers <<a href="mailto:herddb-dev@lists.herddb.org" target="_blank">herddb-dev@lists.herddb.org</a>><br>
Subject: Re: [Herddb-dev] HerdDB collections in production<br>
Message-ID:<br>
<<a href="mailto:CACcefgdrWML5jn0TBa0Wwb_s4MCKWYva_CFVRvVebR1krkzsxQ@mail.gmail.com" target="_blank">CACcefgdrWML5jn0TBa0Wwb_s4MCKWYva_CFVRvVebR1krkzsxQ@mail.gmail.com</a>><br>
Content-Type: text/plain; charset="utf-8"<br>
<br>
Il gio 22 ago 2019, 17:47 Alessandro Luccaroni - Diennea <<br>
<a href="mailto:alessandro.luccaroni@diennea.com" target="_blank">alessandro.luccaroni@diennea.com</a>> ha scritto:<br>
<br>
> Hi list,<br>
><br>
> I?m happy to say that, since the last update of Herd (v0.11.9), we are<br>
> using HerdDB collections in production system and we are very pleased with<br>
> the performance and the memory usage of Herd.<br>
><br>
<br>
<br>
Great news!<br>
<br>
Kudos<br>
<br>
<br>
<br>
><br>
> We first tried to switch with herd v0.11.0 but we encountered some<br>
> problems on services with lots of data being continuisly load and unloaded<br>
> (mainly related to filesystem usage)?you can see an history of the fixes<br>
> needed in the release histo fo the v0.11 branch, thanks Enrico and Diego!<br>
><br>
><br>
> I don?t have hard numbers on hand (I need to switch back to MapDB and<br>
> perform some comparisons, I will update you tomorrow on that), but the<br>
> performance are at least in line (I think they are better) with the added<br>
> benefit of the controlled memory usage and a BIG reduction in storage IOPS<br>
> (Herd data storage versus the memory mapped files of MapDB).<br>
><br>
><br>
><br>
> PS: @Amit I?ve found this benchmark used in past by MapDB (<br>
> <a href="https://github.com/jankotek/mapdb-benchmarks" rel="noreferrer" target="_blank">https://github.com/jankotek/mapdb-benchmarks</a>), maybe it would be good to<br>
> integrate some bench regarding Collections before starting advertise this<br>
> feature of herd<br>
><br>
Sure<br>
We can wait for 0.12.0 release<br>
<br>
<br>
Enrico<br>
<br>
<br>
><br>
> *Alessandro Luccaroni*<br>
> Platform Manager @ Diennea - MagNews<br>
> Tel.: (+39) 0546 066100 Int. 924 - Mob.: (+39) 393 7273519<br>
> Viale G.Marconi 30/14 - 48018 Faenza (RA) - Italy<br>
><br>
><br>
><br>
> ------------------------------<br>
><br>
> CONFIDENTIALITY & PRIVACY NOTICE<br>
> This e-mail (including any attachments) is strictly confidential and may<br>
> also contain privileged information. If you are not the intended recipient<br>
> you are not authorised to read, print, save, process or disclose this<br>
> message. If you have received this message by mistake, please inform the<br>
> sender immediately and destroy this e-mail, its attachments and any copies.<br>
> Any use, distribution, reproduction or disclosure by any person other than<br>
> the intended recipient is strictly prohibited and the person responsible<br>
> may incur in penalties.<br>
> The use of this e-mail is only for professional purposes; there is no<br>
> guarantee that the correspondence towards this e-mail will be read only by<br>
> the recipient, because, under certain circumstances, there may be a need to<br>
> access this email by third subjects belonging to the Company.<br>
> _______________________________________________<br>
> herddb-dev mailing list<br>
> <a href="mailto:herddb-dev@lists.herddb.org" target="_blank">herddb-dev@lists.herddb.org</a><br>
> <a href="http://lists.herddb.org/mailman/listinfo/herddb-dev" rel="noreferrer" target="_blank">http://lists.herddb.org/mailman/listinfo/herddb-dev</a><br>
><br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <<a href="http://lists.herddb.org/pipermail/herddb-dev/attachments/20190822/038a4022/attachment-0001.html" rel="noreferrer" target="_blank">http://lists.herddb.org/pipermail/herddb-dev/attachments/20190822/038a4022/attachment-0001.html</a>><br>
<br>
------------------------------<br>
<br>
Message: 3<br>
Date: Fri, 23 Aug 2019 09:36:44 +0000<br>
From: Alessandro Luccaroni - Diennea<br>
<<a href="mailto:alessandro.luccaroni@diennea.com" target="_blank">alessandro.luccaroni@diennea.com</a>><br>
To: Herddb developers <<a href="mailto:herddb-dev@lists.herddb.org" target="_blank">herddb-dev@lists.herddb.org</a>><br>
Subject: [Herddb-dev] R: HerdDB collections in production<br>
Message-ID: <<a href="mailto:a3ea1bbbd91246a18b7de95c48545101@diennea.com" target="_blank">a3ea1bbbd91246a18b7de95c48545101@diennea.com</a>><br>
Content-Type: text/plain; charset="utf-8"<br>
<br>
Some update on the numbers: I?ve run the same activity in production on MapDB v3.0.7 (with memory mapped files) and HerdDB v0.11.9 and we have a ~13x speedup (6200 sec vs 450 sec)<br>
<br>
All this with slightly lower CPU usage, same memory footprint (with the advantage that all the memory is inside the JVM heap, so we don?t see OOM error from the OS anymore) and a bit more GC load (this is expected since we don?t use direct memory anymore).<br>
<br>
Ale<br>
<br>
Da: <a href="mailto:herddb-dev-bounces@lists.herddb.org" target="_blank">herddb-dev-bounces@lists.herddb.org</a> <<a href="mailto:herddb-dev-bounces@lists.herddb.org" target="_blank">herddb-dev-bounces@lists.herddb.org</a>> Per conto di Enrico Olivelli<br>
Inviato: gioved? 22 agosto 2019 18:39<br>
A: Herddb developers <<a href="mailto:herddb-dev@lists.herddb.org" target="_blank">herddb-dev@lists.herddb.org</a>><br>
Oggetto: Re: [Herddb-dev] HerdDB collections in production<br>
<br>
<br>
We first tried to switch with herd v0.11.0 but we encountered some problems on services with lots of data being continuisly load and unloaded (mainly related to filesystem usage)?you can see an history of the fixes needed in the release histo fo the v0.11 branch, thanks Enrico and Diego!<br>
<br>
I don?t have hard numbers on hand (I need to switch back to MapDB and perform some comparisons, I will update you tomorrow on that), but the performance are at least in line (I think they are better) with the added benefit of the controlled memory usage and a BIG reduction in storage IOPS (Herd data storage versus the memory mapped files of MapDB).<br>
PS: @Amit I?ve found this benchmark used in past by MapDB (<a href="https://github.com/jankotek/mapdb-benchmarks" rel="noreferrer" target="_blank">https://github.com/jankotek/mapdb-benchmarks</a>), maybe it would be good to integrate some bench regarding Collections before starting advertise this feature of herd<br>
Sure<br>
We can wait for 0.12.0 release<br>
<br>
<br>
Enrico<br>
<br>
________________________________<br>
<br>
CONFIDENTIALITY & PRIVACY NOTICE<br>
This e-mail (including any attachments) is strictly confidential and may also contain privileged information. If you are not the intended recipient you are not authorised to read, print, save, process or disclose this message. If you have received this message by mistake, please inform the sender immediately and destroy this e-mail, its attachments and any copies. Any use, distribution, reproduction or disclosure by any person other than the intended recipient is strictly prohibited and the person responsible may incur in penalties.<br>
The use of this e-mail is only for professional purposes; there is no guarantee that the correspondence towards this e-mail will be read only by the recipient, because, under certain circumstances, there may be a need to access this email by third subjects belonging to the Company.<br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <<a href="http://lists.herddb.org/pipermail/herddb-dev/attachments/20190823/c087ab4d/attachment.html" rel="noreferrer" target="_blank">http://lists.herddb.org/pipermail/herddb-dev/attachments/20190823/c087ab4d/attachment.html</a>><br>
<br>
------------------------------<br>
<br>
_______________________________________________<br>
herddb-dev mailing list<br>
<a href="mailto:herddb-dev@lists.herddb.org" target="_blank">herddb-dev@lists.herddb.org</a><br>
<a href="http://lists.herddb.org/mailman/listinfo/herddb-dev" rel="noreferrer" target="_blank">http://lists.herddb.org/mailman/listinfo/herddb-dev</a><br>
<br>
<br>
End of herddb-dev Digest, Vol 14, Issue 7<br>
*****************************************<br>
</blockquote></div>