Permalink
Commits on Sep 20, 2016
  1. Updated TraceGenerator to correctly log machine id on TaskScheduled.

    Affected modules: src/misc
    
    Change-Id: Ib44b733c0dd590c2cd74e98aef764b36d13693ce
    ICGog committed Sep 20, 2016
  2. Split LICENSE file into core and dependencies.

    This refactor has two benefits:
     1) The GitHub license badge should work correctly (previously, it was
        confused by the somewhat homebaked license file contents).
     2) It clearly separates the LICENSE file for the main project (under
        Apache 2.0 license), the dependencies (under various permissive
        licenses), and the solvers (various licenses).
    
    Change-Id: Ifbeb1534a23ae78bff41c89bad7bda27c63f7053
    ms705 committed Sep 20, 2016
  3. Extended trace generator to log machine rack assignments.

    Affected modules: src/misc src/sim/dfs
    
    Change-Id: Id4201c3e4938aa890522b7325799bb146e2d523b
    ICGog committed Sep 20, 2016
  4. Cosmetic fix in README.

    Remove superfluous row in scheduling policy table.
    
    Change-Id: I032a0f828a2359a72a2d498e058a3aebd367a8c2
    ms705 committed Sep 20, 2016
Commits on Sep 19, 2016
  1. Updated net stats usage to differentiate between rx and tx.

    Previously, we were using a single value for network bandwidth.
    This diff changes how we account for network usage by using
    two values: one for network rx and another one for network tx.
    The diff also appropriately updated the CoCo and Net cost models.
    
    Affected modules: scripts src/base src/engine src/platforms
    src/scheduling/flow src/sim src/webui
    
    Change-Id: I30c52be480df35260d16c01101273db85514f435
    ICGog committed Sep 19, 2016
Commits on Sep 6, 2016
  1. Updated network mix script to submit CPU spin tasks as well.

    The workload now consists of 1/3 HDFS get tasks and
    2/3 CPU spin tasks.
    
    Affected modules: scripts/job/benchmark
    
    Change-Id: Id03647d2f614e977fb57d072c10f51caff689956
    ICGog committed Sep 6, 2016
Commits on Sep 5, 2016
  1. Updated benchmark scripts to work with the new Mesos version.

    The new Mesos executor doesn't have an instances flag. This
    change updates the scripts so that they submit a job for
    every task.
    
    Affected modules: scripts/job/benchmark
    
    Change-Id: I0f3515769c53ea8b8147c19d7ce0fdf0648d6bd7
    ICGog committed Sep 5, 2016
Commits on Sep 3, 2016
  1. Updated the cluster network mix script.

    This change comments out the lineitem tasks from the cluster
    network mix script in order to reduce the network load generated
    by the experiment.
    
    Affected modules: scripts/job/benchmark
    
    Change-Id: I8170a78a0a09b013455e146a9dedbc025898971a
    ICGog committed Sep 3, 2016
Commits on Jul 23, 2016
  1. Correctly create directories for example outputs.

    This fixes an issue introduced by
    3a84f6c: when doing
    an out-of-tree build, the target directories for the
    example binaries do not exist, so the build system has
    to create them.
    
    Change-Id: Iccc1fab659d3525fec315b4b6bd7bf64819e1961
    ms705 committed Jul 23, 2016
Commits on Jun 30, 2016
  1. Remove superfluous examples Makefiles.

    We now use CMake for the example programs as well.
    
    Change-Id: I4245ab6cbbe18d32a748dfe26ffa24f50ad3bcd7
    ms705 committed Jun 28, 2016
  2. Disambiguate example target names on in-tree build

    When building within the source tree, the names of the executables in
    the examples/ directory clashed with the names of the subdirectories
    containing the source for the corresponding examples. This change moves
    the binaries into the per-example directories to avoid this clash.
    
    This led to our Docker build failing (see #45), since it is an in-tree
    build.
    
    Issue: #45
    Affected modules: examples
    
    Change-Id: I86588d4ac43dfddce147e2c32ac5d4b17057587e
    ms705 committed Jun 28, 2016
  3. Workaround for CMake issue with in-source builds.

    When building external dependencies inside the source tree, CMake gets
    confused on incremental builds (at which point the dependency's source
    tree is dirty). To work around this issue, we disable dynamic update
    steps for the in-source build dependencies (cs2 and pb2json). In the
    long run, we should port them to support out-of-tree builds.
    
    Change-Id: Ib192de6899ef705f9f13ec4ad6ac863f4bfc9ba3
    ms705 committed Jun 28, 2016
  4. Fail if Quincy cost model used with a DLM.

    This change adds a flag validator for the --enable_hdfs_data_locality
    flag, which guards against the Quincy cost model being used on a
    cluster that does not have the HDFS integration enabled.
    
    Issue: #46
    Affected modules: engine scheduling scheduling/flow storage
    
    Change-Id: I81682e102eff1645114d4878f3f34ed6bfdd1049
    ms705 committed Jun 28, 2016
Commits on Jun 28, 2016
  1. Correctly initialise DLM field if no DLM present.

    The KnowledgeBase keeps a pointer to the DataLayerManager (DLM) around,
    but a KB can be constructed without a DLM. In this case, the
    zero-argument constructor calls into the single-argument constructor
    with NULL as the DLM argument. However, invoking a constructor like this
    does not process its C++ field initialiser statements (as the current
    constructor supposedly has already done so), so data_layer_manager_ was
    actually undefined memory, rather than NULL.
    
    Affected modules: scheduling
    
    Change-Id: Icd237043a5f3b88059e2c25cfedab3a07173a0ab
    ms705 committed Jun 28, 2016
  2. Flag to control if HDFS data locality is enabled.

    Previously, when compiled with ENABLE_HDFS set, Firmament would always
    try to connect an HDFS NameNode. If it could not connect, it failed
    hard. This change gates the HDFS integration behind a new boolean flag
    (--hdfs_data_locality_enabled) even if compiled with ENABLE_HDFS. The
    flag is false by default, so that Firmament works even in environments
    where HDFS isn't available. Fixes #46.
    
    Issue: #46
    Affected modules: engine storage
    
    Change-Id: I9da990e973f09af03d4e310b634a6d223c0f3b47
    ms705 committed Jun 28, 2016
  3. Do not enable HDFS support by default.

    Change-Id: I0c4b58ec50bc92271fcd2d4b5ff97afa4c13e4c0
    ICGog committed Jun 28, 2016
Commits on May 25, 2016
  1. Workaround for pb2json incremental build error.

    CMake gets confused by pb2json's build state inside the source tree.
    Hence, we clean up before trying to build again. This is a bit of a hack
    because it disables the implicit "git pull", but it will fix the
    Poseidon build issues for now.
    ms705 committed May 25, 2016
  2. Add missing dependency to fix build failure.

    Travis (rightly) fails because "storage" does not depend on
    "scheduling". This change adds the missing dependency.
    
    Change-Id: Icd4e114a7136116ce368fdb8af323f3d57be0a68
    ms705 committed May 25, 2016
Commits on May 24, 2016
  1. Add cs2 as libfirmament dependency.

    This ensures that cs2 is downloaded and compiled even when only the
    Firmament integration library is built.
    
    Change-Id: I10f86d907caaf679156e855bbeb574e9dece9dab
    ms705 committed May 24, 2016
  2. Added scheduling method that returns deltas.

    Affected modules: scheduling scheduling/flow scheduling/simple
    
    Change-Id: Iec6466084d4b83099a63aa838b0beeed60cb8b95
    ICGog committed May 24, 2016
Commits on May 16, 2016
  1. Always remove runnning tasks from runnable_tasks set.

    Previously we were only removing delegated tasks from
    the runnable set.
    
    Affected modules: scheduling
    
    Change-Id: Id984f3319acd423a628fc3afc23b4201ca73cd11
    ICGog committed May 16, 2016
  2. Update cost model list in README.

    All of them are working and complete now, plus we've added the net cost
    model.
    
    Change-Id: I0c11971a7d6483a78924e86d6576c2ced3bcbf25
    ms705 committed May 16, 2016
  3. Fix Travis-CI badge.

    Change-Id: If4df5687367402d174d85bb7a33d65bdd824669c
    ms705 committed May 16, 2016
Commits on May 13, 2016
  1. Simple scheduler now correctly updates the runnable_tasks set.

    Affected modules: scheduling scheduling/simple
    
    Change-Id: I62ccd74ec3d2090dd14766aa9421dc983c5f495d
    ICGog committed May 12, 2016
Commits on May 10, 2016
  1. Updated SimpledScheduler to be able to schedule on random resources.

    Affected modules: scheduling/simple
    
    Change-Id: I704e2f20a3b2d1a90e6bb5f7616f957e2314c697
    ICGog committed May 10, 2016
  2. Updated cluster net mix workload.

    Change-Id: Iba008bbb6d3804303a329a7dafdab63eda4c7ba5
    ICGog committed May 10, 2016
  3. Flush task_events after every write.

    Affected modules: misc
    
    Change-Id: Icf8a292d4873236164782e2bde08c1f644589cd8
    ICGog committed May 10, 2016
Commits on May 9, 2016
  1. Updated hdfs network workload generator.

    Change-Id: I2daadd3f83bd8778153c97195dfa4ae5f1f60416
    ICGog committed May 9, 2016
  2. Updated cluster mix network script.

    Change-Id: I2bf9e6d4c78d8ccf1218f466e90cb6e34d8475b4
    ICGog committed May 9, 2016
  3. Do not delete state for completed tasks in the NetCostModel.

    Affected modules: scheduling/flow
    
    Change-Id: Ic2545ec06b7ca2ea486da143291a4c93e041e8fe
    ICGog committed May 9, 2016
  4. Added cluster mix network workload.

    Change-Id: Iee9b566808128c4d21ba5861783e4cde60166d70
    ICGog committed May 9, 2016
  5. Updated the simple scheduler to clear running tasks.

    Affected modules: scheduling scheduling/simple
    
    Change-Id: Ic8b4c6df019953e557e66b5326521f68a97f6695
    ICGog committed May 9, 2016
Commits on May 8, 2016
  1. Extended Net cost model with support for "multi-arcs".

    Affected modules: misc scheduling/flow
    
    Change-Id: Ice5c9035e2dd5b15d2bcc2cb8f203eaafb900304
    ICGog committed May 8, 2016