New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reproducible builds: Non deterministic order of embedded migrations #1901

Closed
kpcyrd opened this Issue Oct 31, 2018 · 0 comments

Comments

Projects
None yet
1 participant
@kpcyrd
Contributor

kpcyrd commented Oct 31, 2018

Setup

I'm trying to make a program pass reprotest for reproducible-builds. I've noticed that embedded_migrations introduce randomness into the binary that may prevent successful rebuilding:

reprotest -vv --vary=-time,-domain_host,-build_path --store-dir artifacts/ --source-pattern 'Cargo.* src/ migrations/' '
    CARGO_HOME="$PWD/.cargo" RUSTUP_HOME='"$HOME/.rustup"' \
        RUSTFLAGS="--remap-path-prefix=$HOME=/remap-home --remap-path-prefix=$PWD=/remap-pwd" \
        cargo build --release --verbose --locked' \
    target/release/fsord

This builds a small example project that has a few migrations embedded:

extern crate diesel;
#[macro_use] extern crate diesel_migrations;

use diesel::sqlite::*;
use diesel::Connection;

embed_migrations!();

fn main() {
    let path = "foo.db";
    let db = SqliteConnection::establish(&path).unwrap();
    embedded_migrations::run(&db).unwrap();
    println!("Hello, world!");
}

This currently fails since the embed_migrations! macro depends on the dirent order, which may be undeterministic:
https://github.com/diesel-rs/diesel/blob/master/diesel_migrations/migrations_internals/src/lib.rs#L264-L279

#[doc(hidden)]
pub fn migration_paths_in_directory(path: &Path) -> Result<Vec<DirEntry>, MigrationError> {
    try!(path.read_dir())
        .filter_map(|entry| {
            let entry = match entry {
                Ok(e) => e,
                Err(e) => return Some(Err(e.into())),
            };
            if entry.file_name().to_string_lossy().starts_with('.') {
                None
            } else {
                Some(Ok(entry))
            }
        })
        .collect()
}

migrations

Versions

  • diesel: 1.3.3
  • diesel_migrations: 1.3.0

Feature Flags

  • diesel: sqlite

Suggested solution

The list of migrations should be sorted, either directly in migration_paths_in_directory, or in the embed_migrations! macro.

I assume this list needs to be sorted at runtime anyway to ensure migrations are applied in the correct order.

Checklist

  • I have already looked over the issue tracker for similar issues.
  • This issue can be reproduced on Rust's stable channel. (Your issue will be
    closed if this is not the case)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment