Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use of Jini #20

Closed
nsoft opened this issue Mar 23, 2016 · 16 comments
Closed

Use of Jini #20

nsoft opened this issue Mar 23, 2016 · 16 comments
Milestone

Comments

@nsoft
Copy link
Owner

nsoft commented Mar 23, 2016

This issue will be used to explain what it is we hope to get out of Jini, why it looks useful and discuss whether or not these expectations are realistic. Also alternative solutions may be examined. There is repeated concern about the health/viability/future of JINI and so folks ivolved in the Apache River project will be invited to comment on this issue and argue for or against these proposed usages of Jini.

General Goals:

  • No-configuration(1) necessary for a node to join a JesterJ system
  • No configuration(1) necessary for nodes to be recognized and controlled by the control web app
  • Distributed, automatically available repository for transfer of documents among nodes.
  • Secure communications among nodes and with the control app.

(1) Except a group name (as per DJ.1.4.1) denoting a particular logical jesterj system (several might exist on the same network), and a secret/password/key/??/etc for access to the system

Specific benefits of Jini:

  • Broadcast: A Multicast Announcement protocol to by which nodes can make their Lookup Service known. Specifically JesterJ ingest nodes will broadcast services to be consumed by other nodes, and to be consumed by the "controler" web application. Services will be things like "I can tell you how to connect to the embedded cassandra" "I can tell you how to join our cassandra cluster (if you are an unaffiliated node with proper credentials)" "I can provide a list of plans, steps and their statuses", I can accept, start or stop a Plan (usually from the control app)"
  • Discovery: The flip side of the above broadcast... A Multicast Request protocol (see Discovery and Join) to find a Lookup Service allowing prospective clients to find their desired services.
  • JavaSpaces: A means of passing java objects among applications in an asynchronous push to space, pull from space manner, with support for expiring the lease on the space if the objects are not picked up in a timely manner. Documents will be pushed to the JavaSpace and their fault tolerant indexing record will include the expiration time of their record's lease so that if they are not handled in time they can be detected, warnings can be issued and the scanner can re-fetch and begin processing the document again.
  • Security: From the spec:The design of the security model for Jini technology is built on the twin notions of a principal and an access control list. Jini services are accessed on behalf of some entity--the principal--which generally traces back to a particular user of the system. Services themselves may request access to other services based on the identity of the object that implements the service. Whether access to a service is allowed depends on the contents of an access control list that is associated with the object. This sounds like what I want. Based on section DJ.3.4 The net.jini.discovery.ssl Format of the Discovery specification it also appears that there should be good support for SSL transport.
@nsoft nsoft added this to the 0.3 milestone Mar 23, 2016
@dgoldenberg1234
Copy link
Contributor

What are some of the alternatives? Should we research some of the latest microservices stacks to achieve these goals?

For both Cassandra and JINI, my concerns would be tight coupling to specific providers of the functionality we need. Could these be refactored out to be behind an interface, with Spring or Guice driving the DI? Perhaps such interfaces are not easy to refactor out.

@nsoft nsoft changed the title Use of JINI Use of Jini Mar 23, 2016
@nsoft
Copy link
Owner Author

nsoft commented Mar 23, 2016

Invitation sent to user@river.apache.org

Hi Folks,

Many years ago I had a friend who spent some time telling me about "how cool" Jini was, but I never really got around to investigating it for myself at that time. Several years ago I was thinking of a project and remembered what he sad and read the Jini specs. They struck me as something that should have been much more successful than they were. They really look like they solve a lot of key problems, and I started playing a little bit and decided I probably wanted to use river. However in the way that side projects often do, it fell victim to work schedules etc. Lately, I've been reviving the project and now I've got a collaborator and a very rudimentary 0.1 release. The primary concern I've heard every time I mention Jini to anyone is the perception that it's old, done, yesterday, dying etc. Having been lurking on this list for the last couple years I can say that releases still happen and it's "Not Dead Yet" (in my best Monty Python voice) but I also can't honestly say it's very lively either. Help me counter the nay sayers :)

I've laid out my reasoning for why it fits with what I want to do in the link below and I'd like to invite anyone who actually uses Jini regularly to do any of the following:

  1. Confirm that my concept/ideas/goals fit well with Jini
  2. Tell me I'm crazy and I don't get it :) (nicely)
  3. Say something somewhere in between 1 and 2...
  4. Encourage me to use Jini
  5. Discourage me from using Jini
  6. Propose alternative technologies
  7. Help me build the Jini parts of my project :)

I've started an issue in my Github repository to describe the way I plan to use Jini and host discussion about pros/cons of this choice so if you're interested you can find it here: #20

I also hope that if this project succeeds it might serve to re-publicize jini a bit and bring it back into the attention sphere of some of the larger open source software community...

Best,

Gus

@nsoft
Copy link
Owner Author

nsoft commented Mar 23, 2016

Ah besides the tutorial link on the river site, google just revealed to me this more detailed site also by Jan Newmarch http://www.javacoffeebreak.com/books/extracts/jini/jini.html

@greggwon
Copy link

If you perform interface based design and programming practices, then you already have what you need, in your design/system, to use Jini. The "Remote" interface is an additional requirement so that all methods in your interface throw "RemoteException". Some people choose to use a different object as a delegate for the remote implementation/inbound-call-sync.

What is important to understand about Jini is the freedom it gives you to stop having binary->text->binary marshal->unmarshal->instantiate overhead in your application. You can just use serializable objects and make the majority of applications work.

What doesn't work so well in any network application, is the transport of large data objects continuously. If you are doing Documents and using Javaspaces as an async store and forward queue, it is probably worth creating a document store service and just providing GUIDs in you java spaces entries which allow any client, at any point, to go get the document from the storage service. That would be the place for your leasing to go through for lifecycle management. The lease time can be stored at the time of insert into the document storage service. Then, you can use a classic timeout mechanism to discard the document. The document storage system could then retain the registration without the document for a more extended time and respond to "Get" requests with an appropriate failure message.

@nsoft
Copy link
Owner Author

nsoft commented Mar 26, 2016

Thanks for the input Gregg. I'll keep your comments about interfaces and serialized objects in mind.

I'm interested in your comments about JavaSpaces. First off, I agree that data transfer has to be minimized. In my conception of the system (which I probably haven't properly documented anywhere) the basic mode of operation is passing documents from step to step in the JVM with no persistence, and of course no network transport. JavaSpaces would only come into play when documents need to be passed from one machine to another. I'm generally thinking that the last step to execute on Machine A before the hop to Machine B would store the Document in a JavaSpace hosted on Machine A, if possible within the same JVM (I hope that's possible but haven't validated). Thus the only cost for that step is expressing the document such that it complies with the Jini Entry Specification. Since we are sending from machine to machine and the Document Processors will often need to access the bytes of the document itself (i.e. we are hopping to one of N machines deployed to provide OCR) we probably can't escape the transport cost of a single transfer per machine hop. Writing to an external store, passing a reference and then reading back from it would seem be two network transfers.

Your point about marshal/unmarshal steps does seem applicable to this however. It's possibly difficult to make the document object itself a valid Entry, and if so we have a similar conversion to/from an Entry (I tried once and gave up but I no longer remember why). Such conversion shouldn't be as heavy as conversion to JSON or XML or database write, but still might be excess processing. The reason steps don't just send to each other directly is that I want to have the steps pull their next document rather than block the preceding step, which is why I need some sort of storage between machine hops. Usually if one has configured the system to hop to a new machine the next step is something heavy that needs to be spread across machines. Writing this I realize that I may not yet have designed in sufficient capacity in this regard between steps within the JVM however. A buffering scheme should probably be be available there too. hmmm... maybe. Need to think more on that. Presently there is a "batchSize" which controls the size of the incoming queues, but really large queues would be memory hogs and when small queues fill up they have to block to avoid loss. Maybe everything should go into the local JavaSpace and I should just make that as efficient as possible?

@dgoldenberg1234
Copy link
Contributor

I'd like to understand the pros and cons of using JINI especially in the light of the recent surge of activity around the notion of microservices and the related tools and frameworks.

For instance, Lightbend, the makers of Scala and Akka, have published a high level overview here. Their CTO Jonas Boner has some interesting presentations on slideshare also.

Other links:

And of course, there's a bunch of materials from Martin Fowler.

I think it may be worth doing some research to see what advantages we may get from JINI vis a vis these other tools and frameworks.

@pfirmstone
Copy link

Jini has been moderately successful, it's used in a number of successful commercial products, but it isn't very visible publicly. It's used for federating compute resources on local area networks, NAT - network address translation limits it's use outside local area networks, however IPv6 doesn't have this limitation. I've registered IPv6 multicast discovery protocols on IANA, but they're not implemented in any Apache River releases yet.

No modern frameworks provide equivalent functionality at this time.

You're interested in SSL, so it sounds like you want to venture past the firewall :)

Regarding security, the SSL protocols haven't been updated, so you can't use the latest and greatest such as AES_256_GCM or key exchanges such as ECDHE_ECDSA and ECDHE_RSA, at least not without modifying the source code.

I'd recommend using Java 8, as this prevents the use of weaker SSL protocols that are vulnerable to MITM attack. The strong encryption constraints in Jini are considered weak nowadays.

Set the following system property:
-Dorg.apache.river.jeri.ssl.sslProtocol=TLSv1.2

Regarding Entry's, these are like a Java interface (prior to Java 8), once defined and deployed, do not change them; or you will end up in a world of hurt. Entry's are not designed for evolution or change, do not change the names of fields, or their order, or add or remove fields from an Entry after deployment. If you need new fields, create a child class that extends the original Entry.

@dgoldenberg1234
Copy link
Contributor

Hi Peter,
Thank you for your detailed response. Very helpful and puts things in perspective.

  • Dmitry

@pfirmstone
Copy link

You're welcome.

If you're new to Apache River and don't have an existing deployment, you may like to try checking out and building from trunk. I believe River is only making source releases going forward, unless someone volunteers to make a binary.

The trunk branch uses Groovy for configuration and contains over 250 bug fixes (most relate to race conditions and Java memory model compliance, a lot of code in 2.2 pre dates the current JMM), the 2.2 code has been in deployment for ten years or so (containing only minor changes since Jini 2.1). However concerns remain that new bugs may have been introduced in trunk and their impact on existing deployments is unknown. Existing deployments have either found workarounds for existing bugs or aren't affected by them, the upcoming River 3.0.0 source release will be in beta, trunk is frozen ready for release, when it occurs.

If you're a new user, you're likely to be less impacted by bugs with trunk, which has been tested on IBM's J9 JVM and Oracles JVM on multiple CPU architectures as well as FindBugs static analysis, it's test suites are all maintained, including the jtreg test suite. The 2.2 branch, has the benefit of existing deployments, however its jtreg test suite hasn't been maintained, certs have expired etc, and it hasn't received the scrutiny that trunk has in recent times.

If you're really adventurous,
you can try my beta binary release called River-Internet 3.0.0-b1 on github, it includes modern secure TLS cypher's, IPv6 Discovery, with global announcement, atomic input validation for Serialization, service authentication before download utilizing two new Java 8 default interface ServiceRegistrar methods and secure discovery providers, ServiceDiscoveryManager has been updated to utilise these new methods. This isn't an Apache release but hopefully the changes it contains will one day be considered for inclusion in Apache River or inspire similar changes.

Cheers,

Peter.

@pfirmstone
Copy link

Regarding IPv4 NAT translation, these are the limitations it places on Jini:

  • Multicast discovery packets are dropped by firewalls / NAT
  • While http and https JERI endpoints may be used for clients to contact publicly available services and a ServiceRegistrar containing them may be provided, clients behind NAT can only be contacted by the server, after the client has initiated the connection and the connection is maintained. Event notification etc doesn't work, when the server tries to cold call the client. However NAT also places similar constraints on SOAP / REST, or protocol buffer based services.
  • IPv6 will allow Jini services, multicast discovery and events to behave on the internet, similar to how they would on a LAN. I don't believe this functionality has been created for SOAP/REST or protocol buffers at this time, but may be later when enabled by IPv6. Although only the Jini multicast announcement protocol would be global, as request is only suitable for the LAN; it would take a few minutes to discover all available lookup services.

@nsoft
Copy link
Owner Author

nsoft commented Mar 31, 2016

IPV6 certainly is something we should be supporting. Is there a way to limit how far away JINI will look (via a TTL on the packets for example?) It would probably be quite cool to run with a TTL of 2 or some small number and be robust vs small network adjustments, but throw up a warning when we see things change... so the system doesn't break but you know your sysadmins are up to something and might need to be reminded of your system's particular needs. Probably over thinking it though :)

@pfirmstone
Copy link

The good thing about IPv6 is addresses are self auto discoverable/self assigning, all the IPv4 network configuration issues dissappear. Although IPv6 supports TTL, additionally, there are multicast groups, eg jini-announcement; node local FF01::155, link local FF02::155, site local FF05::155 and variable scope FF0X::155. Groups are independant, if you join one, you won't receive packets from one of the other groups, unless you join them also.

For now, I've only implemented site local for jini request and site local or variable scope for jini announcement.

@nsoft nsoft modified the milestones: 0.3, 2.0 Sep 11, 2017
@nsoft
Copy link
Owner Author

nsoft commented Jan 14, 2023

Unfortunately Apache River has moved to the attic, and is retired. Some other solution will be required.

@nsoft nsoft closed this as completed Jan 14, 2023
@pfirmstone
Copy link

pfirmstone commented Jan 15, 2023 via email

@nsoft
Copy link
Owner Author

nsoft commented Jan 15, 2023

Hi Peter,

Thanks for the info! You might want to get your project listed on the attic page for river then (https://attic.apache.org/projects/river.html)

@pfirmstone
Copy link

pfirmstone commented Jan 16, 2023 via email

@nsoft nsoft modified the milestones: 2.0, 1.0 Feb 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants