New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for multiline regex in file.replace #28174

Merged
merged 6 commits into from Oct 22, 2015

Conversation

Projects
None yet
2 participants
@lorengordon
Contributor

lorengordon commented Oct 21, 2015

  • use mmap to avoid reading the entire file into real memory
  • mmap treats the entire file as a single string
    • re functions work when using the multiline flag
    • count parameter works as expected
    • no need to read line-by-line; should be significantly faster

Fixes #7999
Fixes #8051

@lorengordon

This comment has been minimized.

Show comment
Hide comment
@lorengordon

lorengordon Oct 21, 2015

Contributor

The issues were marked as bugs, so I developed against the 2015.5 branch. There may be some conflicts merging this forward, as there were some changes to file.replace for 2015.8.

Contributor

lorengordon commented Oct 21, 2015

The issues were marked as bugs, so I developed against the 2015.5 branch. There may be some conflicts merging this forward, as there were some changes to file.replace for 2015.8.

@lorengordon

This comment has been minimized.

Show comment
Hide comment
@lorengordon

lorengordon Oct 21, 2015

Contributor

Also, there is one use case I might have broken...per this note...

#NOTE: Is there a use case for searching an empty file with an empty pattern?
Contributor

lorengordon commented Oct 21, 2015

Also, there is one use case I might have broken...per this note...

#NOTE: Is there a use case for searching an empty file with an empty pattern?

lorengordon added some commits Oct 21, 2015

Add support for multiline regex in file.replace
* use mmap to avoid reading the entire file into real memory
* mmap treats the entire file as a single string
    * `re` functions work when using the multiline flag
    * `count` parameter works as expected
    * no need to read line-by-line; should be significantly faster

Fixes #7999
Fixes #8051
@cachedout

This comment has been minimized.

Show comment
Hide comment
@cachedout

cachedout Oct 21, 2015

Contributor

@lorengordon A number of tests failed here. Could you please take a look?

Contributor

cachedout commented Oct 21, 2015

@lorengordon A number of tests failed here. Could you please take a look?

@lorengordon

This comment has been minimized.

Show comment
Hide comment
@lorengordon

lorengordon Oct 21, 2015

Contributor

@cachedout, already on it. These last two commits should do it.

Contributor

lorengordon commented Oct 21, 2015

@cachedout, already on it. These last two commits should do it.

lorengordon added some commits Oct 22, 2015

Use a test that makes the extra file read unnecessary
The previous test was:
* `has_changes = result != r_data.read(filesize)`

This test was validating that `result` is not the same as the contents
of the file. The only way for `has_changes` to become true at this
point in code would be if `pattern` exists in the file (`nrepl > 0`),
and if `pattern == repl`.

The new test just checks explicitly for this one condition,
eliminating the extra file read.

cachedout added a commit that referenced this pull request Oct 22, 2015

Merge pull request #28174 from lorengordon/file-replace-multiline
Add support for multiline regex in file.replace

@cachedout cachedout merged commit bdd48c9 into saltstack:2015.5 Oct 22, 2015

5 checks passed

default Merged build finished.
Details
jenkins/salt-pr-clone Salt PR - Clone Repository #10516 — SUCCESS
Details
jenkins/salt-pr-linode-ubuntu14.04-n Salt PR - Linode Ubuntu 14.04 #1581 — SUCCESS
Details
jenkins/salt-pr-lint-n Salt PR - Code Lint #10226 — SUCCESS
Details
jenkins/salt-pr-rs-cent7-n Salt PR - RS CentOS 7 #9055 — SUCCESS
Details

@lorengordon lorengordon deleted the lorengordon:file-replace-multiline branch Oct 25, 2015

jfindlay added a commit to jfindlay/salt that referenced this pull request Nov 6, 2015

jfindlay added a commit to jfindlay/salt that referenced this pull request Nov 6, 2015

@twangboy twangboy referenced this pull request Nov 14, 2015

Closed

Fix file.comment #28888

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment