Interpolate %REACT_APP_...% in public/robots.txt

[davetapley]

Is your proposal related to a problem?

%REACT_APP_...% are interpolated in public/index.html but not in public/robots.txt.

IMHO it would be more consistent to apply this to all files in public/ (see Additional context).

This has also come up on #6568.

Describe the solution you'd like

Be able to use %REACT_APP_...% in (at least) public/robots.txt (but optionally all files in public/).

Describe alternatives you've considered

Injecting the correct values in as part of our deployment process.

Additional context

The docs are little inconsistent on this: The ones for the public folder in general state:

None of the files in public folder get post-processed or minified.

But then the env vars docs state:

You can also access the environment variables starting with REACT_APP_ in the public/index.html.

[heyimalex]

IMHO it would be more consistent to apply this to all files in public/

Our string interpolation is pretty much a regex replace for %REACT_APP_[A-Z_]+%, and while that's probably rare in the wild, it seems like blindly doing it in every file in public is pretty bad? I also don't think it's too confusing that index.html is the only "special" file in public.

I'm not necessarily against this, just thinking out loud. We could have a whitelist that clears robots.txt, but that would be even more confusing I think. Maybe there could be a way to opt-in to variable interpolation via special file extension.

And also, as I said in that other thread, there's really nothing wrong with having a custom post-build script that does this for you. People always seem allergic to it, but it's really very easy to set up and maintain.

[ianschmitz]

I agree with @heyimalex. This is fairly easily solved in most cases as a custom script within your project that is run before the react-scripts build.

Agreed the docs could be cleared up. We'd love a PR to do that if you're up for it!