<div dir="ltr"><div dir="ltr">Hello,<br></div><div dir="ltr"><br></div><div>Thanks for starting the discussion. </div><div>My plan today is to study in detail the current implementation of the protocol we have in herddb and then make decision whether to go grpc route or not for the Go language. I will update the team shortly on my findings.</div><div><br></div><div>-Amit</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Jan 17, 2020 at 1:47 AM <<a href="mailto:herddb-dev-request@lists.herddb.org">herddb-dev-request@lists.herddb.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Send herddb-dev mailing list submissions to<br>
<a href="mailto:herddb-dev@lists.herddb.org" target="_blank">herddb-dev@lists.herddb.org</a><br>
<br>
To subscribe or unsubscribe via the World Wide Web, visit<br>
<a href="http://lists.herddb.org/mailman/listinfo/herddb-dev" rel="noreferrer" target="_blank">http://lists.herddb.org/mailman/listinfo/herddb-dev</a><br>
or, via email, send a message with subject or body 'help' to<br>
<a href="mailto:herddb-dev-request@lists.herddb.org" target="_blank">herddb-dev-request@lists.herddb.org</a><br>
<br>
You can reach the person managing the list at<br>
<a href="mailto:herddb-dev-owner@lists.herddb.org" target="_blank">herddb-dev-owner@lists.herddb.org</a><br>
<br>
When replying, please edit your Subject line so it is more specific<br>
than "Re: Contents of herddb-dev digest..."<br>
<br>
<br>
Today's Topics:<br>
<br>
1. Re: R: Herddb clients with go and python (Enrico Olivelli)<br>
2. R: R: Herddb clients with go and python<br>
(Alessandro Luccaroni - Diennea)<br>
<br>
<br>
----------------------------------------------------------------------<br>
<br>
Message: 1<br>
Date: Fri, 17 Jan 2020 09:09:33 +0100<br>
From: Enrico Olivelli <<a href="mailto:eolivelli@gmail.com" target="_blank">eolivelli@gmail.com</a>><br>
To: Herddb developers <<a href="mailto:herddb-dev@lists.herddb.org" target="_blank">herddb-dev@lists.herddb.org</a>><br>
Subject: Re: [Herddb-dev] R: Herddb clients with go and python<br>
Message-ID:<br>
<CACcefgdSiWB4D=<a href="mailto:B9HY29vmeUEQe3f7fmfnX0YDkWD-kiXu_Xmg@mail.gmail.com" target="_blank">B9HY29vmeUEQe3f7fmfnX0YDkWD-kiXu_Xmg@mail.gmail.com</a>><br>
Content-Type: text/plain; charset="utf-8"<br>
<br>
Alessandro,<br>
<br>
Il giorno ven 17 gen 2020 alle ore 09:04 Alessandro Luccaroni - Diennea <<br>
<a href="mailto:alessandro.luccaroni@diennea.com" target="_blank">alessandro.luccaroni@diennea.com</a>> ha scritto:<br>
<br>
> We don?t have (as far as I know) many ?expert? in languages outside Java<br>
> so I think going down the route of supporting protobuf/grpc is a good<br>
> investment: if we get more traction it?s way easier for a newcomer to<br>
> provide and support a new client this way.<br>
><br>
><br>
><br>
> @Enrico: when you talk about ?performing the Java part? do you mean to<br>
> switch also the Java client to grpc? If the performance are good/acceptable<br>
> I think we should also evaluate to drop the custom binary protocol to<br>
> streamline the code (this should be feasible right now since we have only<br>
> one client implementation and we are still in the growing phase of the<br>
> project)<br>
><br>
<br>
No, I mean to do the plumbing of the gRPC server into the server.<br>
Current "custom" protocol is super optimized in order to save as many in in<br>
memory copies as possible, both in the client and in the server,<br>
when the server receives a request it does need to create intermediate java<br>
objects from the parts of the request, it reads directly from the ByteBuf,<br>
the same is for the client.<br>
For a resultset the client receives a "big" ByteBuf and it reads the values<br>
directly from that buffer.<br>
Any other "higher level" abstract needs to create intermediate Java objects<br>
or add memory copies.<br>
<br>
So by now I don't like to drop current wire protocol.<br>
Maybe I am just biased by the experience with Flatbuffers we had in 0.7.0.<br>
<br>
Enrico<br>
<br>
<br>
><br>
><br>
> Ale<br>
><br>
><br>
><br>
> *Da:* <a href="mailto:herddb-dev-bounces@lists.herddb.org" target="_blank">herddb-dev-bounces@lists.herddb.org</a> <<br>
> <a href="mailto:herddb-dev-bounces@lists.herddb.org" target="_blank">herddb-dev-bounces@lists.herddb.org</a>> *Per conto di *Enrico Olivelli<br>
> *Inviato:* gioved? 16 gennaio 2020 22:02<br>
> *A:* Herddb developers <<a href="mailto:herddb-dev@lists.herddb.org" target="_blank">herddb-dev@lists.herddb.org</a>><br>
> *Oggetto:* Re: [Herddb-dev] Herddb clients with go and python<br>
><br>
><br>
><br>
> Amit<br>
><br>
> Il gio 16 gen 2020, 19:30 Amit Chavan <<a href="mailto:achavan1@gmail.com" target="_blank">achavan1@gmail.com</a>> ha scritto:<br>
><br>
> Hello,<br>
><br>
><br>
><br>
> What does the team think about writing some clients for herddb in other<br>
> languages?<br>
><br>
> We can start with Go or python to begin with? Let me know your thoughts?<br>
><br>
><br>
><br>
> It would be great.<br>
><br>
><br>
><br>
> For network transport do we use anything specific with the current java<br>
> implementation?<br>
><br>
><br>
><br>
> We have a custom protocol, I think you already touched it.<br>
><br>
> If we can support our native binary protocol it would be easier to get the<br>
> same performances as for the java binding.<br>
><br>
><br>
><br>
> We cam also add some alternative protocol using protobuf or even grpc.<br>
><br>
> Maybe with grcp most of the wire protocol work is already done by build<br>
> time plugins.<br>
><br>
><br>
><br>
> If we want to go down the protobuf/grcp way I am happy to do the Java<br>
> part. I know that Diego knows well grcp so maybe he can work on that topic<br>
> as well (can't talk for me btw)<br>
><br>
><br>
><br>
> I will start documenting the discussion and design approaches as we start<br>
> having a discussion on this email thread.<br>
><br>
><br>
><br>
> Very good<br>
><br>
><br>
><br>
><br>
><br>
> Enrico<br>
><br>
><br>
><br>
> -Amit<br>
><br>
> _______________________________________________<br>
> herddb-dev mailing list<br>
> <a href="mailto:herddb-dev@lists.herddb.org" target="_blank">herddb-dev@lists.herddb.org</a><br>
> <a href="http://lists.herddb.org/mailman/listinfo/herddb-dev" rel="noreferrer" target="_blank">http://lists.herddb.org/mailman/listinfo/herddb-dev</a><br>
><br>
><br>
> ------------------------------<br>
><br>
> CONFIDENTIALITY & PRIVACY NOTICE<br>
> This e-mail (including any attachments) is strictly confidential and may<br>
> also contain privileged information. If you are not the intended recipient<br>
> you are not authorised to read, print, save, process or disclose this<br>
> message. If you have received this message by mistake, please inform the<br>
> sender immediately and destroy this e-mail, its attachments and any copies.<br>
> Any use, distribution, reproduction or disclosure by any person other than<br>
> the intended recipient is strictly prohibited and the person responsible<br>
> may incur in penalties.<br>
> The use of this e-mail is only for professional purposes; there is no<br>
> guarantee that the correspondence towards this e-mail will be read only by<br>
> the recipient, because, under certain circumstances, there may be a need to<br>
> access this email by third subjects belonging to the Company.<br>
> _______________________________________________<br>
> herddb-dev mailing list<br>
> <a href="mailto:herddb-dev@lists.herddb.org" target="_blank">herddb-dev@lists.herddb.org</a><br>
> <a href="http://lists.herddb.org/mailman/listinfo/herddb-dev" rel="noreferrer" target="_blank">http://lists.herddb.org/mailman/listinfo/herddb-dev</a><br>
><br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <<a href="http://lists.herddb.org/pipermail/herddb-dev/attachments/20200117/b8d63a5d/attachment-0001.html" rel="noreferrer" target="_blank">http://lists.herddb.org/pipermail/herddb-dev/attachments/20200117/b8d63a5d/attachment-0001.html</a>><br>
<br>
------------------------------<br>
<br>
Message: 2<br>
Date: Fri, 17 Jan 2020 09:46:40 +0000<br>
From: Alessandro Luccaroni - Diennea<br>
<<a href="mailto:alessandro.luccaroni@diennea.com" target="_blank">alessandro.luccaroni@diennea.com</a>><br>
To: Herddb developers <<a href="mailto:herddb-dev@lists.herddb.org" target="_blank">herddb-dev@lists.herddb.org</a>><br>
Subject: [Herddb-dev] R: R: Herddb clients with go and python<br>
Message-ID: <<a href="mailto:298b5eb04aee4a978d2763122fcf26df@diennea.com" target="_blank">298b5eb04aee4a978d2763122fcf26df@diennea.com</a>><br>
Content-Type: text/plain; charset="utf-8"<br>
<br>
I didn?t remember that we tried out Flatbuffers early on (with bad results)...if, indeed, the performance are not in the same ballpark (and, especially, if we hit the memory on the server side) maybe stick to the binary protocol is preferable.<br>
<br>
Alessandro<br>
<br>
Da: <a href="mailto:herddb-dev-bounces@lists.herddb.org" target="_blank">herddb-dev-bounces@lists.herddb.org</a> <<a href="mailto:herddb-dev-bounces@lists.herddb.org" target="_blank">herddb-dev-bounces@lists.herddb.org</a>> Per conto di Enrico Olivelli<br>
Inviato: venerd? 17 gennaio 2020 09:10<br>
A: Herddb developers <<a href="mailto:herddb-dev@lists.herddb.org" target="_blank">herddb-dev@lists.herddb.org</a>><br>
Oggetto: Re: [Herddb-dev] R: Herddb clients with go and python<br>
<br>
Alessandro,<br>
<br>
Il giorno ven 17 gen 2020 alle ore 09:04 Alessandro Luccaroni - Diennea <<a href="mailto:alessandro.luccaroni@diennea.com" target="_blank">alessandro.luccaroni@diennea.com</a><mailto:<a href="mailto:alessandro.luccaroni@diennea.com" target="_blank">alessandro.luccaroni@diennea.com</a>>> ha scritto:<br>
We don?t have (as far as I know) many ?expert? in languages outside Java so I think going down the route of supporting protobuf/grpc is a good investment: if we get more traction it?s way easier for a newcomer to provide and support a new client this way.<br>
<br>
@Enrico: when you talk about ?performing the Java part? do you mean to switch also the Java client to grpc? If the performance are good/acceptable I think we should also evaluate to drop the custom binary protocol to streamline the code (this should be feasible right now since we have only one client implementation and we are still in the growing phase of the project)<br>
<br>
No, I mean to do the plumbing of the gRPC server into the server.<br>
Current "custom" protocol is super optimized in order to save as many in in memory copies as possible, both in the client and in the server,<br>
when the server receives a request it does need to create intermediate java objects from the parts of the request, it reads directly from the ByteBuf, the same is for the client.<br>
For a resultset the client receives a "big" ByteBuf and it reads the values directly from that buffer.<br>
Any other "higher level" abstract needs to create intermediate Java objects or add memory copies.<br>
<br>
So by now I don't like to drop current wire protocol.<br>
Maybe I am just biased by the experience with Flatbuffers we had in 0.7.0.<br>
<br>
Enrico<br>
<br>
<br>
Ale<br>
<br>
Da: <a href="mailto:herddb-dev-bounces@lists.herddb.org" target="_blank">herddb-dev-bounces@lists.herddb.org</a><mailto:<a href="mailto:herddb-dev-bounces@lists.herddb.org" target="_blank">herddb-dev-bounces@lists.herddb.org</a>> <<a href="mailto:herddb-dev-bounces@lists.herddb.org" target="_blank">herddb-dev-bounces@lists.herddb.org</a><mailto:<a href="mailto:herddb-dev-bounces@lists.herddb.org" target="_blank">herddb-dev-bounces@lists.herddb.org</a>>> Per conto di Enrico Olivelli<br>
Inviato: gioved? 16 gennaio 2020 22:02<br>
A: Herddb developers <<a href="mailto:herddb-dev@lists.herddb.org" target="_blank">herddb-dev@lists.herddb.org</a><mailto:<a href="mailto:herddb-dev@lists.herddb.org" target="_blank">herddb-dev@lists.herddb.org</a>>><br>
Oggetto: Re: [Herddb-dev] Herddb clients with go and python<br>
<br>
Amit<br>
Il gio 16 gen 2020, 19:30 Amit Chavan <<a href="mailto:achavan1@gmail.com" target="_blank">achavan1@gmail.com</a><mailto:<a href="mailto:achavan1@gmail.com" target="_blank">achavan1@gmail.com</a>>> ha scritto:<br>
Hello,<br>
<br>
What does the team think about writing some clients for herddb in other languages?<br>
We can start with Go or python to begin with? Let me know your thoughts?<br>
<br>
It would be great.<br>
<br>
For network transport do we use anything specific with the current java implementation?<br>
<br>
We have a custom protocol, I think you already touched it.<br>
If we can support our native binary protocol it would be easier to get the same performances as for the java binding.<br>
<br>
We cam also add some alternative protocol using protobuf or even grpc.<br>
Maybe with grcp most of the wire protocol work is already done by build time plugins.<br>
<br>
If we want to go down the protobuf/grcp way I am happy to do the Java part. I know that Diego knows well grcp so maybe he can work on that topic as well (can't talk for me btw)<br>
<br>
I will start documenting the discussion and design approaches as we start having a discussion on this email thread.<br>
<br>
Very good<br>
<br>
<br>
Enrico<br>
<br>
-Amit<br>
_______________________________________________<br>
herddb-dev mailing list<br>
<a href="mailto:herddb-dev@lists.herddb.org" target="_blank">herddb-dev@lists.herddb.org</a><mailto:<a href="mailto:herddb-dev@lists.herddb.org" target="_blank">herddb-dev@lists.herddb.org</a>><br>
<a href="http://lists.herddb.org/mailman/listinfo/herddb-dev" rel="noreferrer" target="_blank">http://lists.herddb.org/mailman/listinfo/herddb-dev</a><br>
<br>
________________________________<br>
<br>
CONFIDENTIALITY & PRIVACY NOTICE<br>
This e-mail (including any attachments) is strictly confidential and may also contain privileged information. If you are not the intended recipient you are not authorised to read, print, save, process or disclose this message. If you have received this message by mistake, please inform the sender immediately and destroy this e-mail, its attachments and any copies. Any use, distribution, reproduction or disclosure by any person other than the intended recipient is strictly prohibited and the person responsible may incur in penalties.<br>
The use of this e-mail is only for professional purposes; there is no guarantee that the correspondence towards this e-mail will be read only by the recipient, because, under certain circumstances, there may be a need to access this email by third subjects belonging to the Company.<br>
_______________________________________________<br>
herddb-dev mailing list<br>
<a href="mailto:herddb-dev@lists.herddb.org" target="_blank">herddb-dev@lists.herddb.org</a><mailto:<a href="mailto:herddb-dev@lists.herddb.org" target="_blank">herddb-dev@lists.herddb.org</a>><br>
<a href="http://lists.herddb.org/mailman/listinfo/herddb-dev" rel="noreferrer" target="_blank">http://lists.herddb.org/mailman/listinfo/herddb-dev</a><br>
<br>
________________________________<br>
<br>
CONFIDENTIALITY & PRIVACY NOTICE<br>
This e-mail (including any attachments) is strictly confidential and may also contain privileged information. If you are not the intended recipient you are not authorised to read, print, save, process or disclose this message. If you have received this message by mistake, please inform the sender immediately and destroy this e-mail, its attachments and any copies. Any use, distribution, reproduction or disclosure by any person other than the intended recipient is strictly prohibited and the person responsible may incur in penalties.<br>
The use of this e-mail is only for professional purposes; there is no guarantee that the correspondence towards this e-mail will be read only by the recipient, because, under certain circumstances, there may be a need to access this email by third subjects belonging to the Company.<br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <<a href="http://lists.herddb.org/pipermail/herddb-dev/attachments/20200117/b055ae80/attachment.html" rel="noreferrer" target="_blank">http://lists.herddb.org/pipermail/herddb-dev/attachments/20200117/b055ae80/attachment.html</a>><br>
<br>
------------------------------<br>
<br>
_______________________________________________<br>
herddb-dev mailing list<br>
<a href="mailto:herddb-dev@lists.herddb.org" target="_blank">herddb-dev@lists.herddb.org</a><br>
<a href="http://lists.herddb.org/mailman/listinfo/herddb-dev" rel="noreferrer" target="_blank">http://lists.herddb.org/mailman/listinfo/herddb-dev</a><br>
<br>
<br>
End of herddb-dev Digest, Vol 18, Issue 8<br>
*****************************************<br>
</blockquote></div></div>