Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Field Modules #217

Closed
11 tasks done
jgaehring opened this issue Jun 30, 2019 · 15 comments
Closed
11 tasks done

Field Modules #217

jgaehring opened this issue Jun 30, 2019 · 15 comments

Comments

@jgaehring
Copy link
Member

jgaehring commented Jun 30, 2019

Summary

Field modules are highly customizable application modules that can be loaded at runtime, based on information from the server. Each of these modules would function almost like a small application in its own right, but would share some common structure with the core app. This is similar to the concept of micro frontends. The user would be able to install, enable and disable these field modules from the farmOS admin panel, so each farm could easily decide which field modules were pertinent to their own operations. This way we avoid feature bloat and cluttering the UI with unnecessary options, while still being capable of treating highly specific use cases.

The basic architecture for the client would consist of a core application, which would represent an app shell, a shared model for syncing and persisting data, and a mechanism for fetching information from the server regarding what field modules were available and where their corresponding scripts could be found. It would also need a way to register these scripts at runtime and integrate them with the app shell. Key to this operation would be some kind of manifest, which each module would register with the server, and which could be passed along to the client via a REST endpoint (eg, /farm.json). This manifest would have a unique name and/or id for each field module, plus the url where the main script for that module could be found. The server would also do it’s part in hosting that script from a CORS-enabled URL.

History

The concept of field modules is one we’ve been kicking around for a while, going back at least as far as issue #38. It's also been discussed in issue #123, in the proposed "New Architecture", and this gist. I’ve recently submitted a PR (#216) for a prototype of how these field modules might be loaded at runtime, and I think we're finally close enough to start thinking of concrete steps we can take towards a full implementation.

Development Requirements

Other Features & Development

Server Development Requirements

  • serve the modules’ scripts
  • serve the module data at /farm.json
@jgaehring
Copy link
Member Author

Got a specific question I'd like your input on @mstenta .... What kind of correspondence do you think there should be between field modules and farmOS modules? Should each field module belong to a specific farmOS module? Or should the farmOS module just be one of the field module's dependencies? Can a farmOS module house multiple field modules? What if a field module has dependencies on multiple farmOS modules? Perhaps field modules should be wrapped up as their own separate farmOS modules entirely?

I tend to think that last option may be the best, to keep the amount of coupling low and enable them to take on multiple dependencies, but I wondered what your thoughts were on this.

This will be crucial, I believe, to how we structure the data at /farm.json

@mstenta
Copy link
Member

mstenta commented Jul 1, 2019

What kind of correspondence do you think there should be between field modules and farmOS modules? Should each field module belong to a specific farmOS module? Or should the farmOS module just be one of the field module's dependencies?

Yes, the way I'm thinking of it, field modules would be "housed" in a farmOS (Drupal) module (perhaps with some exceptions - like maybe the app comes with a few default ones itself?).

It might help for me to sketch up an example of what this looks like in my mind... I might try to do that after this and paste a link...

Housing them in farmOS modules has a number of benefits:

  1. Allows the farmOS server maintainer to decide which farmOS modules to enable as step 1, and which field modules to enable as step 2 (see additional thoughts on this below).
  2. Leverages the existing Drupal dependency management to define and enforce which server modules the field module depends on.
  3. Greatly simplifies the farmOS-client codebase - and moves the responsibility for field module code to the farmOS modules that provide them. These may be in the farmOS core codebase (https://github.com/farmOS/farmOS), or in add-on modules that are maintained in other repositories (eg: the weather module https://github.com/farmOS/farm_weather, the eggs module https://github.com/farmOS/farm_eggs, etc)
  4. As described in Pull translations from the farmOS server #214, I hope we can "push" translated strings from the server to the client somehow - so that all the translations can be managed in one place (https://localize.drupal.org/translate/projects/farm)

Can a farmOS module house multiple field modules?

Yes, this will be easy to do - the farmOS server module will simply implement a new hook that describes each field module it provides - so there won't be any difference in difficulty between providing one or many.

Now, in practice, maybe it's better to only provide one... and maybe we recommend that field modules are the ONLY thing in the farmOS module... "to keep the amount of coupling low" as you said. But that's a convention decision, and not an architectural one, the way I see it. It will be technically possible to include multiple field modules, and do other Drupal stuff, all in the same module, if someone wants to. But maybe we can try to set a good example in farmOS core by keeping things separate and simple.

What if a field module has dependencies on multiple farmOS modules?

If the field module is housed in a Drupal module, then the Drupal module will define what other modules it depends on. See the Eggs module info file for an example: https://github.com/farmOS/farm_eggs/blob/7.x-1.x/farm_eggs.info

Perhaps field modules should be wrapped up as their own separate farmOS modules entirely?

Yea, perhaps (see my thoughts on conventions above). But ultimately I don't think this will be a strict requirement - more of an organizational decision.

Additional thoughts on enabling field modules

I've been having some ideas about how to provide flexibility in enabling/disabling field modules...

As I said above, it can be a two step process, if we want... whereby the farmOS server admin first installs the farmOS (Drupal) module that contains the field module. And then there is a second decision about which field modules to actually send to the app.

There are a few levels of features/complexity we could take with this...

  1. At the simplest, we just make it 1-to-1 - so when a farmOS module is enabled, any field modules it contains are automatically enabled as well and appear in the app. This might be the simplest way for us to start... this could be our MVP.
  2. The next level would be to add an admin UI inside farmOS, which allows a farmOS server admin to enabled/disable individual field modules server-wide. So for example, if a farmOS module provided multiple field modules (again, maybe we don't do that, but for the sake of hypotheticals), then this admin UI would allow the admin to only push one of them to the app.
  3. The ultimate level (until we devise even greater ideas) would be to control which field modules are available in the client on a user-by-user basis. This actually wouldn't be too difficult, and all the logic for it could live in the farmOS server - so the client can remain simple. For example, imagine an admin UI in farmOS that lets you enable field modules on a user-by-user basis (or by role, we can decide). Then, that would affect what field modules appear in /farm.json, so the client would only pull in the ones it sees.

Notably, level 2 above is how "Quick Forms" work in farmOS currently. farmOS modules can provide one or more quick forms, and then the user can decide which ones they actually want to include in their quick form menu.

Level 3 is something that I've heard requests for already - and I think would be really useful. Imagine if you have workers who only have one job. You give them the app, enable a single field module for them, and they are good to go. From their perspective, they only need to learn that one module's UI, and never even see anything else. Phew! That would be great! :-D

@mstenta
Copy link
Member

mstenta commented Jul 1, 2019

Here is a work-in-progress commit that demonstrates how we might convert the weather quick form to a field module:

https://github.com/farmOS/farm_weather/compare/field_module

It basically does three things:

  1. Deletes the PHP quick form (in farm_weather.farm_quick.inc), along with the hook_help() function in farm_weather.module (which was only needed for the quick form).
  2. Implements a new hook_farm_client_modules() function (tentative name), which simply returns an array describing what field modules are included (currently just a name and path to JS file):
/**
 * Implements hook_farm_client_modules().
 */
function farm_weather_farm_client_modules() {
  return array(
    'weather' => array(
      'label' => t('Weather'),
      'js' => drupal_get_path('module', 'farm_weather') . 'src/FieldModule/Weather/weather.js',
    ),
  );
}
  1. Adds a JS file src/FieldModule/Weather/weather.js which is where the field module code would be. This is just a placeholder, and if we need multiple files that's also doable.

@mstenta
Copy link
Member

mstenta commented Jul 1, 2019

Implements a new hook_farm_client_module() function

The other side to this would be some code in farmOS that invokes that hook (calls the function in all enabled module) to gather info about field modules, and then puts the info into /farm.json (along with any additional logic we want for enabling/disabling field modules on a user-by-user basis).

@mstenta
Copy link
Member

mstenta commented Jul 1, 2019

Notably, level 2 above is how "Quick Forms" work in farmOS currently. farmOS modules can provide one or more quick forms, and then the user can decide which ones they actually want to include in their quick form menu.

Here is what this looks like in farmOS currently:

Screenshot from 2019-07-01 10-10-07

So ultimately, if we move all quick forms into field modules, we can convert that form to a "Field Module Config" form perhaps.

Note that in this screenshot, there are 7 quick forms available, but only 5 are enabled. "Enabled" just means that they appear as sub-tabs under the "Quick" tab on the dashboard.

Notably, some of those quick forms are provided by a single farmOS module. Here is the breakdown:

  • Livestock module (in farmOS core) provides "Birth" and "Milk".
  • Crop module (in farmOS core) provides "Planting".
  • Soil module (in farmOS core) provides "Soil amendment" and "Soil disturbance".
  • Eggs module (separate add-on module) provides "Eggs".
  • Weather module (separate add-on module) provides "Weather".

So you can see what I mean about modules potentially providing multiple quick forms (or field modules), and still having the option of enabling/disabling them individually.

I keep chickens, so I have the Livestock module enabled, but I don't breed or keep goats/cows, so I don't enable the "Birth" or "Milk" quick forms. Now, maybe those should be in separate modules from Livestock... we can reconsider where they live if/when we convert them to field modules.

@mstenta
Copy link
Member

mstenta commented Jul 1, 2019

Alright, I pushed a new farm_client branch to my fork of farmOS: farmOS/farmOS@7.x-1.x...mstenta:farm_client

Super simple... just adds a new farm_client module to farmOS, which provides hook_farm_client_modules() as described above.

This, combined with the Weather module's field_module branch should be enough to get you started!

Just checkout farm_client branch in your local farmOS, download and checkout the field_module branch of the Weather module (https://github.com/farmOS/farm_weather), and then install both modules in Drupal (/admin/modules).

You should then see the following in your /farm.json endpoint:

Screenshot from 2019-07-01 10-42-25

@jgaehring
Copy link
Member Author

jgaehring commented Jul 1, 2019

I'll nitpick one thing...

It would be nicer if modules was an array of objects, and each object had a name value, instead of modules being an object with named properties. Like so:

"client": {
  "modules": [
    {
      "name": "weather",
      "label": "Weather",
      "js": "sites/all/modules/farm_weather/src/FieldModule/Weather/weather.js"
    }
  ]
}

Would be nice if resources were the same. Just feels more JSON-y, and makes it easier to iterate over (and filter etc) via JS array methods.

@jgaehring
Copy link
Member Author

Woohoo!! I got field modules running from my local farmOS Docker server!

There's one hitch, though... It only works when I use a relative path to the script, and that only works because Webpack is running a proxy server. If I use the full path, I get a CORS error. We'll need to set headers on the script for Access-Control-Allow-Origin: * or include the url that Field Kit is being served from. This shouldn't be an issue for the native app, but will be for the web implementation.

@mstenta
Copy link
Member

mstenta commented Jul 2, 2019

It would be nicer if modules was an array of objects, and each object had a name value, instead of modules being an object with named properties.

Done! I amended the wip commits on the farmOS farm_client branch and the Weather field_module branch.

We'll need to set headers on the script for Access-Control-Allow-Origin: * or include the url that Field Kit is being served from.

Hmm this is trickier than the cases we've solved before (eg: allowing access to the API endpoints). In this case, Drupal won't be serving the JS paths itself, but rather allowing the web server (Apache, Nginx, etc, whatever is being used) to serve them as normal files. This means Drupal doesn't have any opportunity to add headers. That will have to happen at the server level.

That's not too hard to do, but it definitely introduces an additional impediment to self-hosting. We could package the config into the Docker file, but then people would have to either use Docker or replicate the config themselves on their server. Not everyone will have access to that server-level config, too, depending on their host.

One option we could consider: DO allow Drupal to serve those scripts, rather than serving them directly from the filesystem. This feels like using a sledgehammer to hang a picture, though.

Hmm.

@mstenta
Copy link
Member

mstenta commented Jul 2, 2019

One option we could consider: DO allow Drupal to serve those scripts, rather than serving them directly from the filesystem. This feels like using a sledgehammer to hang a picture, though.

Copying this from chat with @jgaehring - describing what this would look like:

We have the farm_client module provide a menu router path, at eg: /farm/client/script/*
And when that path is accessed, Drupal would parse * and return the content of whatever script was requested
And add headers
The scripts would ALSO be available via the filesystem directly, at eg: /sites/all/modules/farm_weather/src/FieldModule/index.js or whatever (without the headers)

@jgaehring
Copy link
Member Author

I just realized one thing we'll need to do as a part of the Drupal module that wraps the Weather field module is to add proper units for recording precipitation, eg, inches and/or centimeters.

@mstenta, I imagine there's a fairly standard way of doing this with Drupal, no?

@jgaehring
Copy link
Member Author

Oh, and also the 'weather' category.

@mstenta
Copy link
Member

mstenta commented Nov 21, 2019

Yes! Good point - the weather module should create the units that it needs so they are available.

It already creates the "Weather" category: https://github.com/farmOS/farm_weather/blob/7.x-1.x/farm_weather.module#L23

This will then get synced into Field Kit by virtue of the Weather field module's filter config. One thing to keep in mind, that we talked about in chat: there is still the possibility that a farmOS admin could edit/delete the "Weather" category or the quantity unit terms we create. So to avoid potential bugs caused by that edge case, Field Kit would probably still want to check that the terms existed, and create them if they didn't. Maybe a PUT request would be useful in this case, since the intention of PUT is to create new if it doesn't exist, or update existing if it does.

@mstenta
Copy link
Member

mstenta commented Jul 24, 2020

FYI I merged the field_modules branch into farmOS, so farmOS modules can now serve Field Module JS files from the server: farmOS/farmOS@04d5280

@jgaehring
Copy link
Member Author

I think we're just about ready to merge #216! I removed a couple non-critical items from the checklist that I don't think we're going to get to for a while yet, so I'm going to close this now. Field Module API development will continue to be a high priority as outlined in the Roadmap, #373, but I think what we have right now represents a pretty good MVP.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

No branches or pull requests

2 participants