New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support nanoc environments #859

Merged
merged 10 commits into from Aug 22, 2016

Conversation

Projects
None yet
4 participants
@barraq
Contributor

barraq commented May 31, 2016

Description

Introduce support for nanoc environment and addresses #676.

  • nanoc command is updated to include --env argument
  • site config is overriden using the active environment if any and fallback to 'default'
  • if environments is provided, tmp directory is updated to tmp/{{env_name}}

Setting environments is done in nanoc.yaml using the environments property.
Example usage is:

output_dir: output

environments:
  default: &default
    base_url: ...
    ...
  development:
    <<: *default
    base_url: ...
  production:
    <<: *default
    base_url: ...
  yet_another_env:
    <<: *default
    base_url: ...
    output_dir: build

Selecting working environment can be done:

  • using environment variable NANOC_ENVIRONMENT
  • using nanoc --env=[<value>]

Progress

  • Main Nanoc command (cli/commands/nanoc.rb) is updated to include --env option
  • Nanoc::Int::Configuration is updated to provide with_environment(env = nil)
  • unless define in environment, output_dir is set as {{output_dir}}/{{env_name}}, if environments is not defined then output_dir remains unchanged
  • if environments is provided, tmp directory is updated to tmp/{{env_name}}
  • New tests are added

@barraq barraq changed the title from Support nanoc environments to [WIP] Support nanoc environments May 31, 2016

@barraq

This comment has been minimized.

Show comment
Hide comment
@barraq

barraq May 31, 2016

Contributor

Is missing:

  • handeling /tmp dir so that we use the same mechanism as the output_dir.
  • unit tests

Might be added:

  • introduce @env (@site.env) variable to access only active env variable
Contributor

barraq commented May 31, 2016

Is missing:

  • handeling /tmp dir so that we use the same mechanism as the output_dir.
  • unit tests

Might be added:

  • introduce @env (@site.env) variable to access only active env variable
@gpakosz

This comment has been minimized.

Show comment
Hide comment
@gpakosz

gpakosz May 31, 2016

Member

Does it take into account the cascading of configuration files (parent_config_file)?

Member

gpakosz commented May 31, 2016

Does it take into account the cascading of configuration files (parent_config_file)?

@gpakosz

This comment has been minimized.

Show comment
Hide comment
@gpakosz

gpakosz May 31, 2016

Member

Is <<: *default an inheritance mechanism?

Member

gpakosz commented May 31, 2016

Is <<: *default an inheritance mechanism?

@barraq

This comment has been minimized.

Show comment
Hide comment
@barraq

barraq May 31, 2016

Contributor

Good point regarding parent_config_file. Actually the config is overridden by environment after it is loaded, therefore after it includes the parent_config_file: should be 🆗 then.
Regarding the <<: *default it is just some YAML property for inheritance indeed: https://en.wikipedia.org/wiki/YAML#References. You don't need to use it, I just did cause it is common practice.

Contributor

barraq commented May 31, 2016

Good point regarding parent_config_file. Actually the config is overridden by environment after it is loaded, therefore after it includes the parent_config_file: should be 🆗 then.
Regarding the <<: *default it is just some YAML property for inheritance indeed: https://en.wikipedia.org/wiki/YAML#References. You don't need to use it, I just did cause it is common practice.

@ddfreyne

This comment has been minimized.

Show comment
Hide comment
@ddfreyne

ddfreyne May 31, 2016

Member

This looks cool! A couple of things that I spotted so far:

  • The #site_from_config method takes the configuration as an argument, and modifies it. I dislike modifying incoming arguments, because it makes code harder to follow and effects more difficult to trace.
  • Whatever’s in lib/nanoc/base can’t refer to anything in lib/nanoc/cli; the command-line interface knows about base, but not vice versa.
Member

ddfreyne commented May 31, 2016

This looks cool! A couple of things that I spotted so far:

  • The #site_from_config method takes the configuration as an argument, and modifies it. I dislike modifying incoming arguments, because it makes code harder to follow and effects more difficult to trace.
  • Whatever’s in lib/nanoc/base can’t refer to anything in lib/nanoc/cli; the command-line interface knows about base, but not vice versa.

@barraq barraq changed the title from [WIP] Support nanoc environments to [MVP] Support nanoc environments Jun 7, 2016

@barraq barraq changed the title from [MVP] Support nanoc environments to Support nanoc environments Jun 9, 2016

@barraq

This comment has been minimized.

Show comment
Hide comment
@barraq

barraq Jun 11, 2016

Contributor

@ddfreyne do you need me to do any additional changes?

Contributor

barraq commented Jun 11, 2016

@ddfreyne do you need me to do any additional changes?

@ddfreyne

This comment has been minimized.

Show comment
Hide comment
@ddfreyne

ddfreyne Jun 18, 2016

Member

@barraq I’ll give this a more thorough review soon!

Member

ddfreyne commented Jun 18, 2016

@barraq I’ll give this a more thorough review soon!

@ddfreyne ddfreyne added to review and removed status:wip 🔧 labels Jun 18, 2016

@ddfreyne ddfreyne added this to the 4.3 milestone Jun 19, 2016

@ddfreyne

This comment has been minimized.

Show comment
Hide comment
@ddfreyne

ddfreyne Jun 19, 2016

Member

Looks good otherwise! Looking forward to using this.

Member

ddfreyne commented Jun 19, 2016

Looks good otherwise! Looking forward to using this.

@ddfreyne ddfreyne added status:wip 🔧 and removed to review labels Jun 19, 2016

@barraq

This comment has been minimized.

Show comment
Hide comment
@barraq

barraq Jun 19, 2016

Contributor

Thanks @ddfreyne for this deep review. I fixed all but one:

I’m not in favour of modifying output_dir like this. The output_dir can be specified on a per-environment basis, if necessary, so there’s no need for this logic.

Actually I did this to prevent having a mandatory argument when using environments. To my understanding if the user does not override the output_dir in each defined environment then there will be some weird things happening with the compilation when --prune is activated for instance? Same if the option is not activated since residues from another compilation might remain. I mean that line prevents side effect behavior introduced by environments.

What do you think?

I am fine with removing it :D I just prefer having a second thought before doing it.

One last think; should this PR include the documentation with it? How to do link new feature and the Nanoc.ws documentation?

p.s. sorry for all the parenthesis thingy; I relied on Rubocop.

Contributor

barraq commented Jun 19, 2016

Thanks @ddfreyne for this deep review. I fixed all but one:

I’m not in favour of modifying output_dir like this. The output_dir can be specified on a per-environment basis, if necessary, so there’s no need for this logic.

Actually I did this to prevent having a mandatory argument when using environments. To my understanding if the user does not override the output_dir in each defined environment then there will be some weird things happening with the compilation when --prune is activated for instance? Same if the option is not activated since residues from another compilation might remain. I mean that line prevents side effect behavior introduced by environments.

What do you think?

I am fine with removing it :D I just prefer having a second thought before doing it.

One last think; should this PR include the documentation with it? How to do link new feature and the Nanoc.ws documentation?

p.s. sorry for all the parenthesis thingy; I relied on Rubocop.

@ddfreyne

This comment has been minimized.

Show comment
Hide comment
@ddfreyne

ddfreyne Jun 25, 2016

Member

I’m short on time atm, but will revisit as soon as I can.

Member

ddfreyne commented Jun 25, 2016

I’m short on time atm, but will revisit as soon as I can.

@ddfreyne ddfreyne added to review and removed status:wip 🔧 labels Jun 25, 2016

@ddfreyne

This comment has been minimized.

Show comment
Hide comment
@ddfreyne

ddfreyne Jul 10, 2016

Member

Thanks @ddfreyne for this deep review. I fixed all but one:

I’m not in favour of modifying output_dir like this. The output_dir can be specified on a per-environment basis, if necessary, so there’s no need for this logic.

Actually I did this to prevent having a mandatory argument when using environments. To my understanding if the user does not override the output_dir in each defined environment then there will be some weird things happening with the compilation when --prune is activated for instance? Same if the option is not activated since residues from another compilation might remain. I mean that line prevents side effect behavior introduced by environments.

What do you think?

I am fine with removing it :D I just prefer having a second thought before doing it.

Compiling with one env and pruning with another will cause the pruning to happen as if the site was compiled with the env used for pruning. I think that’s expected behavior.

The current implementation (which sets output_dir) is not backwards compatible. Output will be written to output/default/ by default, rather than output/, which is a change in behavior.

Even retaining the current behavior for the default env is problematic, as output will be written to either output/ or a subdirectory (e.g. output/devel/ when using the devel env), which is even more confusing.

Member

ddfreyne commented Jul 10, 2016

Thanks @ddfreyne for this deep review. I fixed all but one:

I’m not in favour of modifying output_dir like this. The output_dir can be specified on a per-environment basis, if necessary, so there’s no need for this logic.

Actually I did this to prevent having a mandatory argument when using environments. To my understanding if the user does not override the output_dir in each defined environment then there will be some weird things happening with the compilation when --prune is activated for instance? Same if the option is not activated since residues from another compilation might remain. I mean that line prevents side effect behavior introduced by environments.

What do you think?

I am fine with removing it :D I just prefer having a second thought before doing it.

Compiling with one env and pruning with another will cause the pruning to happen as if the site was compiled with the env used for pruning. I think that’s expected behavior.

The current implementation (which sets output_dir) is not backwards compatible. Output will be written to output/default/ by default, rather than output/, which is a change in behavior.

Even retaining the current behavior for the default env is problematic, as output will be written to either output/ or a subdirectory (e.g. output/devel/ when using the devel env), which is even more confusing.

end
it 'has the default environment custom option' do
expect(subject[:foo]).to eq('default-bar')

This comment has been minimized.

@ddfreyne

ddfreyne Jul 10, 2016

Member

Add a test for options defined not within the default env, but globally (should return bar)

@ddfreyne

ddfreyne Jul 10, 2016

Member

Add a test for options defined not within the default env, but globally (should return bar)

@ddfreyne ddfreyne added status:wip 🔧 and removed to review labels Jul 10, 2016

@ddfreyne

This comment has been minimized.

Show comment
Hide comment
@ddfreyne

ddfreyne Jul 24, 2016

Member

@barraq Is this PR still being worked on? I can take over if you wish.

Member

ddfreyne commented Jul 24, 2016

@barraq Is this PR still being worked on? I can take over if you wish.

@gpakosz

This comment has been minimized.

Show comment
Hide comment
@gpakosz

gpakosz Jul 27, 2016

Member

BTW, how does this feature interact with nanoc view?

Member

gpakosz commented Jul 27, 2016

BTW, how does this feature interact with nanoc view?

@ddfreyne ddfreyne modified the milestones: 4.3, 4.4 Aug 21, 2016

barraq added some commits May 31, 2016

Support nanoc environments
Introduce basic support for nanoc environment:
- nanoc command is updated to include --env argument
- config are overriden using the active environment if any
- unless define in environment, output_dir is {{output_dir}}/{{env_name}}

Setting environments is done in nanoc.yaml using the `environments` property.
Example usage is:

```
output_dir: output

environments:
  default: &default
    base_url: ...
    ...
  development:
    <<: *default
    base_url: ...
  production:
    <<: *default
    base_url: ...
  yet_another_env:
    <<: *default
    base_url: ...
    output_dir: build
```

Selecting working environment can be done:
- using environment variable `NANOC_ENVIRONMENT`
- using `nanoc --env=[<value>]`
fixup! Denis review
- Document env attr_reader
- Only allow String and nil for environment name
- Remove argument for with_environment
- Make :environments a constant
- Lot of parentheses fix
- Renamed NANOC_ENVIRONMENT to NANOC_ENV for consistency with Rails, Rake, etc.
- Prefer Fetch(a,b) over a||b
- Refactor tmp_path logic into Nanoc::Int::Store
- Make -e, --env argument required
- Updated test according to previous changes
fixup! improve store
- add Contracts
- rename store to store_name
- use named arguments
@barraq

This comment has been minimized.

Show comment
Hide comment
@barraq

barraq Aug 21, 2016

Contributor

@ddfreyne I went over all fixes you mentioned. I did small fixup commits to ease the review process.
@gpakosz support for environments allows you to override the @site.config. You can then use it however you want: e.g. filter (within your Rules file) draft items in production environment; display additional things in templates for a given environments; etc.

Contributor

barraq commented Aug 21, 2016

@ddfreyne I went over all fixes you mentioned. I did small fixup commits to ease the review process.
@gpakosz support for environments allows you to override the @site.config. You can then use it however you want: e.g. filter (within your Rules file) draft items in production environment; display additional things in templates for a given environments; etc.

@ddfreyne

This comment has been minimized.

Show comment
Hide comment
@ddfreyne

ddfreyne Aug 22, 2016

Member

Looks good! Thanks for this feature. Merging!

Member

ddfreyne commented Aug 22, 2016

Looks good! Thanks for this feature. Merging!

@ddfreyne ddfreyne merged commit d529a87 into nanoc:master Aug 22, 2016

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
@bburton

This comment has been minimized.

Show comment
Hide comment
@bburton

bburton Aug 23, 2016

As much as I like the idea of adding environment support, this change makes certain assumptions that will cause problems with certain filters. For instance, the compass filter creates a .sass-cache folder in the project directory where nanoc compile is run. It's quite possible other filters may create temporary folders where files are cached.

As a result, the only safe way to use this feature is to only compile to one environment in a given project folder and not compile different environments in the same project folder.

Sure it's possible to configure the sass compiler to cache to a specific location but the setup is complicated because the environment setting needs to be passed to the compass config.rb and used to set the sass :cache_location property. Not only that, the configuration has to be made to config.rb so it's possible to run the compass command by itself outside of nanoc and have it pick up the right environment location.

Compass also supports environments so it would be ideal if the environment setting configured for Nanoc could be passed into the compass config.rb but that's a bit beyond the current topic.

However, aside from the filters nanoc supports out of the box, even if all of them were reviewed and fixed if necessary, there's no guarantee that third-party filters aren't writing or caching files to the project directory.

Even if switching environments to compile into the same folder is not supported, environment support is still very useful when building with one environment configuration such as development in its own folder, then push changes to a Git repository, e.g. github.com, then in a different folder configured as the production environment perform a git pull and recompile.

bburton commented Aug 23, 2016

As much as I like the idea of adding environment support, this change makes certain assumptions that will cause problems with certain filters. For instance, the compass filter creates a .sass-cache folder in the project directory where nanoc compile is run. It's quite possible other filters may create temporary folders where files are cached.

As a result, the only safe way to use this feature is to only compile to one environment in a given project folder and not compile different environments in the same project folder.

Sure it's possible to configure the sass compiler to cache to a specific location but the setup is complicated because the environment setting needs to be passed to the compass config.rb and used to set the sass :cache_location property. Not only that, the configuration has to be made to config.rb so it's possible to run the compass command by itself outside of nanoc and have it pick up the right environment location.

Compass also supports environments so it would be ideal if the environment setting configured for Nanoc could be passed into the compass config.rb but that's a bit beyond the current topic.

However, aside from the filters nanoc supports out of the box, even if all of them were reviewed and fixed if necessary, there's no guarantee that third-party filters aren't writing or caching files to the project directory.

Even if switching environments to compile into the same folder is not supported, environment support is still very useful when building with one environment configuration such as development in its own folder, then push changes to a Git repository, e.g. github.com, then in a different folder configured as the production environment perform a git pull and recompile.

@ddfreyne

This comment has been minimized.

Show comment
Hide comment
@ddfreyne

ddfreyne Aug 28, 2016

Member

@bburton Thanks for the comment! I’ve collected your thoughts in #937.

Member

ddfreyne commented Aug 28, 2016

@bburton Thanks for the comment! I’ve collected your thoughts in #937.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment