[Herddb-dev] R: R: Herddb clients with go and python

Alessandro Luccaroni - Diennea alessandro.luccaroni at diennea.com
Fri Jan 17 10:46:40 CET 2020


I didn’t remember that we tried out Flatbuffers early on (with bad results)...if, indeed, the performance are not in the same ballpark (and, especially, if we hit the memory on the server side) maybe stick to the binary protocol is preferable.

Alessandro

Da: herddb-dev-bounces at lists.herddb.org <herddb-dev-bounces at lists.herddb.org> Per conto di Enrico Olivelli
Inviato: venerdì 17 gennaio 2020 09:10
A: Herddb developers <herddb-dev at lists.herddb.org>
Oggetto: Re: [Herddb-dev] R: Herddb clients with go and python

Alessandro,

Il giorno ven 17 gen 2020 alle ore 09:04 Alessandro Luccaroni - Diennea <alessandro.luccaroni at diennea.com<mailto:alessandro.luccaroni at diennea.com>> ha scritto:
We don’t have (as far as I know) many “expert” in languages outside Java so I think going down the route of supporting protobuf/grpc is a good investment: if we get more traction it’s way easier for a newcomer to provide and support a new client this way.

@Enrico: when you talk about “performing the Java part” do you mean to switch also the Java client to grpc? If the performance are good/acceptable I think we should also evaluate to drop the custom binary protocol to streamline the code (this should be feasible right now since we have only one client implementation and we are still in the growing phase of the project)

No, I mean to do the plumbing of the gRPC server into the server.
Current "custom" protocol is super optimized in order to save as many in in memory copies as possible, both in the client and in the server,
when the server receives a request it does need to create intermediate java objects from the parts of the request, it reads directly from the ByteBuf, the same is for the client.
For a resultset the client receives a "big" ByteBuf and it reads the values directly from that buffer.
Any other "higher level" abstract needs to create intermediate Java objects or add memory copies.

So by now I don't like to drop current wire protocol.
Maybe I am just biased by the experience with Flatbuffers we had in 0.7.0.

Enrico


Ale

Da: herddb-dev-bounces at lists.herddb.org<mailto:herddb-dev-bounces at lists.herddb.org> <herddb-dev-bounces at lists.herddb.org<mailto:herddb-dev-bounces at lists.herddb.org>> Per conto di Enrico Olivelli
Inviato: giovedì 16 gennaio 2020 22:02
A: Herddb developers <herddb-dev at lists.herddb.org<mailto:herddb-dev at lists.herddb.org>>
Oggetto: Re: [Herddb-dev] Herddb clients with go and python

Amit
Il gio 16 gen 2020, 19:30 Amit Chavan <achavan1 at gmail.com<mailto:achavan1 at gmail.com>> ha scritto:
Hello,

What does the team think about writing some clients for herddb in other languages?
We can start with Go or python to begin with? Let me know your thoughts?

It would be great.

For network transport do we use anything specific with the current java implementation?

We have a custom protocol, I think you already touched it.
If we can support our native binary protocol it would be easier to get the same performances as for the java binding.

We cam also add some alternative protocol using protobuf or even grpc.
Maybe with grcp most of the wire protocol work is already done by build time plugins.

If we want to go down the protobuf/grcp way I am happy to do the Java part. I know that Diego knows well grcp so maybe he can work on that topic as well (can't talk for me btw)

I will start documenting the discussion and design approaches as we start having a discussion on this email thread.

 Very good


Enrico

-Amit
_______________________________________________
herddb-dev mailing list
herddb-dev at lists.herddb.org<mailto:herddb-dev at lists.herddb.org>
http://lists.herddb.org/mailman/listinfo/herddb-dev

________________________________

CONFIDENTIALITY & PRIVACY NOTICE
This e-mail (including any attachments) is strictly confidential and may also contain privileged information. If you are not the intended recipient you are not authorised to read, print, save, process or disclose this message. If you have received this message by mistake, please inform the sender immediately and destroy this e-mail, its attachments and any copies. Any use, distribution, reproduction or disclosure by any person other than the intended recipient is strictly prohibited and the person responsible may incur in penalties.
The use of this e-mail is only for professional purposes; there is no guarantee that the correspondence towards this e-mail will be read only by the recipient, because, under certain circumstances, there may be a need to access this email by third subjects belonging to the Company.
_______________________________________________
herddb-dev mailing list
herddb-dev at lists.herddb.org<mailto:herddb-dev at lists.herddb.org>
http://lists.herddb.org/mailman/listinfo/herddb-dev

________________________________

CONFIDENTIALITY & PRIVACY NOTICE
This e-mail (including any attachments) is strictly confidential and may also contain privileged information. If you are not the intended recipient you are not authorised to read, print, save, process or disclose this message. If you have received this message by mistake, please inform the sender immediately and destroy this e-mail, its attachments and any copies. Any use, distribution, reproduction or disclosure by any person other than the intended recipient is strictly prohibited and the person responsible may incur in penalties.
The use of this e-mail is only for professional purposes; there is no guarantee that the correspondence towards this e-mail will be read only by the recipient, because, under certain circumstances, there may be a need to access this email by third subjects belonging to the Company.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.herddb.org/pipermail/herddb-dev/attachments/20200117/b055ae80/attachment-0001.html>


More information about the herddb-dev mailing list