Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add NetworkBuilderConfig as part of bundle manifest #644

Open
ansoncfit opened this issue Nov 1, 2020 · 2 comments
Open

Add NetworkBuilderConfig as part of bundle manifest #644

ansoncfit opened this issue Nov 1, 2020 · 2 comments
Assignees

Comments

@ansoncfit
Copy link
Member

ansoncfit commented Nov 1, 2020

Configuration options for bundles could help address a number of outstanding issues:

It might be worth the effort to allow some of these options to be set in analysis requests, but it's probably sufficient for at least some of them to be set once at the network level.

But note that "network bundle creation" (on the backend) is not the same as network building (on workers). It's not enough to specify assumptions about streets when uploading OSM -- those assumptions need to be saved somewhere and passed along to the worker that actually builds the network.

One idea is to save a new NetworkBuilderConfig in the bundle manifest json saved to S3. With that approach, we would no longer rely on passing a default TNBuilderConfig in TransportNetwork.fromFiles().

At first glance, a TNBuilderConfig file is read when building a network...

builderConfig = loadJson(new File(directory, BUILDER_CONFIG_FILENAME));

But in practice, we don't provide that config file, so R5 reverts to the default in this block...

} catch (FileNotFoundException ex) {
LOG.info("File '{}' is not present. Using default configuration.", file);
return TNBuilderConfig.defaultConfig();

This current implementation seems subpar, and many of the fields in TNBuilderConfig are for old point-to-point routing instead of analysis.

We could strip out the fallback to a default config, and instead throw an exception if a bundle manifest does not have the config. There's not a backwards compatibility concern, because old workers will continue to have the fallback.

@ansoncfit
Copy link
Member Author

The configuration options at hand could just be in the top level object, rather than in a nested config within the manifest. And actually this flat top-level object would be a networkBuildConfig rather than a manifest. We can safely eliminate the concept of bundle manifests; in storage, these are just [networkId].json without the terms bundle or manifest hardcoded.

@ansoncfit
Copy link
Member Author

ansoncfit commented Apr 26, 2022

Just discussed the idea of allowing network .dat files to be rewritten -- it might be useful for people to be able to save car egress tables, for example, after they have built a network. But there's a lot to consider here, about workers that might already be running etc.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant