New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Process trace files from Google Cluster Data #149

Closed
manoelcampos opened this Issue Jul 6, 2018 · 0 comments

Comments

Projects
None yet
1 participant
@manoelcampos
Owner

manoelcampos commented Jul 6, 2018

Implement classes to read Google Trace files, enabling the creation of Hosts and Cloudlets from these files.

Detailed information about how the feature should work

New classes should be introduced to implement the format of the different trace files. The existing WorkloalFileReader must be refactored to extract a superclass containing common methods for the other classes.

A brief explanation of why you think this feature is useful

It will enable creating more realistic simulations using real and extensive data from physical datacenters. This data can be assessed, for instance, to identify a possible correlation between different workloads, such as CPU and RAM requirements.

Examples

Related Issues

@manoelcampos manoelcampos added this to the CloudSim Plus 4.0 milestone Jul 6, 2018

@manoelcampos manoelcampos self-assigned this Jul 6, 2018

@manoelcampos manoelcampos changed the title from Process Google Trace files to Process trace files from Google Cluster Data Jul 6, 2018

manoelcampos added a commit that referenced this issue Aug 9, 2018

Closes #149 :: Process trace files from Google Cluster Data
- Refactors WorkloadFileReader extracting methods to a superclasse to enable
  creating new subclasses for different trace formats
  (such as Google Cluster Data)
- Renames WorkloadFileReader to SwfWorkloadFileReader
  because the class is specific for the Standard Workload Format (*.swf)
  from The Hebrew University of Jerusalem.

- Introduces the GoogleMachineEventsTraceReader to create Hosts
  from Google Cloud trace files.

  * Adds startTime and shutdownTime attributes to Host
  * Enables to request the creation of Hosts only at the time
    specified by the timestamp in trace file.
  * Enables processing of ADD and REMOVE event types.
  * Adds GoogleMachineEventsExample1.

- Introduces the GoogleTaskEventsTraceReader
  to read tasks events from Google Cloud traces.
  Jobs files are not read because they don't have useful information
  for simulations and are represented in CloudSim Plus just as an integer
  value for the jobId attribute of a Cloudlet.

  * Adds new Cloudlet status to conform to the Google Cluster Trace tasks events
  * Adds new CloudSimTags to enable a broker to receive Cloudlet status changes
    when creating Cloudlets from a trace file such as the Google Cluster Trace
  * Adds a jobId attribute to Cloudlet so that Cloudlets
    can be categorized as belonging to a fictitious job.
  * Adds DatacenterBroker.getCloudletSubmittedList

- FieldIndex enums in GoogleTraceReaderAbstract subclasses
  enable getting a value from the parsed trace line directly
  from a method inside the enum. This way, the value is returned
  in the correct type and after making unit conversions
  (if required). That was accomplished by making the enum to
  implement the TraceField interface.

- Adds google-cluster-data-samples.xlsx
  as an easier way to analyse the structure of Google Trace files.
- Adds sample google trace files to the resource directory
  of the examples project.

- Fix Conversion issues (such as GB to MB) and updates documentation.
- Changes stream and "enhanced for" in CloudSim class to iterators
  and indexed loops to improve efficiency and avoid
  ConcurrentModificationException.
  Before it was being created defensive copies of event and entities list
  to avoid such an exception, but reducing performance when
  there is a large number of objects in such lists.

Signed-off-by: Manoel Campos <manoelcampos@gmail.com>

manoelcampos added a commit that referenced this issue Aug 9, 2018

Closes #149 :: Process trace files from Google Cluster Data
- Refactors WorkloadFileReader extracting methods to a superclasse to enable
  creating new subclasses for different trace formats
  (such as Google Cluster Data)
- Renames WorkloadFileReader to SwfWorkloadFileReader
  because the class is specific for the Standard Workload Format (*.swf)
  from The Hebrew University of Jerusalem.

- Introduces the GoogleMachineEventsTraceReader to create Hosts
  from Google Cloud trace files.

  * Adds startTime and shutdownTime attributes to Host
  * Enables to request the creation of Hosts only at the time
    specified by the timestamp in trace file.
  * Enables processing of ADD and REMOVE event types.
  * Adds GoogleMachineEventsExample1.

- Introduces the GoogleTaskEventsTraceReader
  to read tasks events from Google Cloud traces.
  Jobs files are not read because they don't have useful information
  for simulations and are represented in CloudSim Plus just as an integer
  value for the jobId attribute of a Cloudlet.

  * Adds new Cloudlet status to conform to the Google Cluster Trace tasks events
  * Adds new CloudSimTags to enable a broker to receive Cloudlet status changes
    when creating Cloudlets from a trace file such as the Google Cluster Trace
  * Adds a jobId attribute to Cloudlet so that Cloudlets
    can be categorized as belonging to a fictitious job.
  * Adds DatacenterBroker.getCloudletSubmittedList
  * Adds GoogleTaskEventsExample1.

- FieldIndex enums in GoogleTraceReaderAbstract subclasses
  enable getting a value from the parsed trace line directly
  from a method inside the enum. This way, the value is returned
  in the correct type and after making unit conversions
  (if required). That was accomplished by making the enum to
  implement the TraceField interface.

- Adds google-cluster-data-samples.xlsx
  as an easier way to analyse the structure of Google Trace files.
- Adds sample google trace files to the resource directory
  of the examples project.

- Fix Conversion issues (such as GB to MB) and updates documentation.
- Changes stream and "enhanced for" in CloudSim class to iterators
  and indexed loops to improve efficiency and avoid
  ConcurrentModificationException.
  Before it was being created defensive copies of event and entities list
  to avoid such an exception, but reducing performance when
  there is a large number of objects in such lists.

Signed-off-by: Manoel Campos <manoelcampos@gmail.com>

manoelcampos added a commit that referenced this issue Aug 9, 2018

Closes #149 :: Process trace files from Google Cluster Data
- Refactors WorkloadFileReader extracting methods to a superclasse to enable
  creating new subclasses for different trace formats
  (such as Google Cluster Data)
- Renames WorkloadFileReader to SwfWorkloadFileReader
  because the class is specific for the Standard Workload Format (*.swf)
  from The Hebrew University of Jerusalem.

- Introduces the GoogleMachineEventsTraceReader to create Hosts
  from Google Cloud trace files.

  * Adds startTime and shutdownTime attributes to Host
  * Enables to request the creation of Hosts only at the time
    specified by the timestamp in trace file.
  * Enables processing of ADD and REMOVE event types.
  * Adds GoogleMachineEventsExample1.

- Introduces the GoogleTaskEventsTraceReader
  to read tasks events from Google Cloud traces.
  Jobs files are not read because they don't have useful information
  for simulations and are represented in CloudSim Plus just as an integer
  value for the jobId attribute of a Cloudlet.

  * Adds new Cloudlet status to conform to the Google Cluster Trace tasks events
  * Adds new CloudSimTags to enable a broker to receive Cloudlet status changes
    when creating Cloudlets from a trace file such as the Google Cluster Trace
  * Adds a jobId attribute to Cloudlet so that Cloudlets
    can be categorized as belonging to a fictitious job.
  * Adds DatacenterBroker.getCloudletSubmittedList
  * Adds GoogleTaskEventsExample1.

- FieldIndex enums in GoogleTraceReaderAbstract subclasses
  enable getting a value from the parsed trace line directly
  from a method inside the enum. This way, the value is returned
  in the correct type and after making unit conversions
  (if required). That was accomplished by making the enum to
  implement the TraceField interface.

- Adds google-cluster-data-samples.xlsx
  as an easier way to analyse the structure of Google Trace files.
- Adds sample google trace files to the resource directory
  of the examples project.

- Fix Conversion issues (such as GB to MB) and updates documentation.
- Changes stream and "enhanced for" in CloudSim class to iterators
  and indexed loops to improve efficiency and avoid
  ConcurrentModificationException.
  Before it was being created defensive copies of event and entities list
  to avoid such an exception, but reducing performance when
  there is a large number of objects in such lists.

Signed-off-by: Manoel Campos <manoelcampos@gmail.com>

manoelcampos added a commit that referenced this issue Aug 9, 2018

Closes #149 :: Process trace files from Google Cluster Data
- Refactors WorkloadFileReader extracting methods to a superclasse to enable
  creating new subclasses for different trace formats
  (such as Google Cluster Data)
- Renames WorkloadFileReader to SwfWorkloadFileReader
  because the class is specific for the Standard Workload Format (*.swf)
  from The Hebrew University of Jerusalem.

- Introduces the GoogleMachineEventsTraceReader to create Hosts
  from Google Cloud trace files.

  * Adds startTime and shutdownTime attributes to Host
  * Enables to request the creation of Hosts only at the time
    specified by the timestamp in trace file.
  * Enables processing of ADD and REMOVE event types.
  * Adds GoogleMachineEventsExample1.

- Introduces the GoogleTaskEventsTraceReader
  to read tasks events from Google Cloud traces.
  Jobs files are not read because they don't have useful information
  for simulations and are represented in CloudSim Plus just as an integer
  value for the jobId attribute of a Cloudlet.

  * Adds new Cloudlet status to conform to the Google Cluster Trace tasks events
  * Adds new CloudSimTags to enable a broker to receive Cloudlet status changes
    when creating Cloudlets from a trace file such as the Google Cluster Trace
  * Adds a jobId attribute to Cloudlet so that Cloudlets
    can be categorized as belonging to a fictitious job.
  * Adds DatacenterBroker.getCloudletSubmittedList
  * Adds GoogleTaskEventsExample1.

- FieldIndex enums in GoogleTraceReaderAbstract subclasses
  enable getting a value from the parsed trace line directly
  from a method inside the enum. This way, the value is returned
  in the correct type and after making unit conversions
  (if required). That was accomplished by making the enum to
  implement the TraceField interface.

- Adds google-cluster-data-samples.xlsx
  as an easier way to analyse the structure of Google Trace files.
- Adds sample google trace files to the resource directory
  of the examples project.

- Fix Conversion issues (such as GB to MB) and updates documentation.
- Changes stream and "enhanced for" in CloudSim class to iterators
  and indexed loops to improve efficiency and avoid
  ConcurrentModificationException.
  Before it was being created defensive copies of event and entities list
  to avoid such an exception, but reducing performance when
  there is a large number of objects in such lists.

Signed-off-by: Manoel Campos <manoelcampos@gmail.com>

manoelcampos added a commit that referenced this issue Aug 10, 2018

Closes #149 :: Process trace files from Google Cluster Data
- Refactors WorkloadFileReader extracting methods to a superclasse to enable
  creating new subclasses for different trace formats
  (such as Google Cluster Data)
- Renames WorkloadFileReader to SwfWorkloadFileReader
  because the class is specific for the Standard Workload Format (*.swf)
  from The Hebrew University of Jerusalem.

- Introduces the GoogleMachineEventsTraceReader to create Hosts
  from Google Cloud trace files.

  * Adds startTime and shutdownTime attributes to Host
  * Enables to request the creation of Hosts only at the time
    specified by the timestamp in trace file.
  * Enables processing of ADD and REMOVE event types.
  * Adds GoogleMachineEventsExample1.

- Introduces the GoogleTaskEventsTraceReader
  to read tasks events from Google Cloud traces.
  Jobs files are not read because they don't have useful information
  for simulations and are represented in CloudSim Plus just as an integer
  value for the jobId attribute of a Cloudlet.

  * Adds new Cloudlet status to conform to the Google Cluster Trace tasks events
  * Adds new CloudSimTags to enable a broker to receive Cloudlet status changes
    when creating Cloudlets from a trace file such as the Google Cluster Trace
  * Adds a jobId attribute to Cloudlet so that Cloudlets
    can be categorized as belonging to a fictitious job.
  * Adds DatacenterBroker.getCloudletSubmittedList
  * Adds GoogleTaskEventsExample1.

- FieldIndex enums in GoogleTraceReaderAbstract subclasses
  enable getting a value from the parsed trace line directly
  from a method inside the enum. This way, the value is returned
  in the correct type and after making unit conversions
  (if required). That was accomplished by making the enum to
  implement the TraceField interface.

- Adds google-cluster-data-samples.xlsx
  as an easier way to analyse the structure of Google Trace files.
- Adds sample google trace files to the resource directory
  of the examples project.

- Fix Conversion issues (such as GB to MB) and updates documentation.
- Changes stream and "enhanced for" in CloudSim class to iterators
  and indexed loops to improve efficiency and avoid
  ConcurrentModificationException.
  Before it was being created defensive copies of event and entities list
  to avoid such an exception, but reducing performance when
  there is a large number of objects in such lists.

Signed-off-by: Manoel Campos <manoelcampos@gmail.com>

manoelcampos added a commit that referenced this issue Aug 10, 2018

Closes #149 :: Process trace files from Google Cluster Data
- Refactors WorkloadFileReader extracting methods to a superclasse to enable
  creating new subclasses for different trace formats
  (such as Google Cluster Data)
- Renames WorkloadFileReader to SwfWorkloadFileReader
  because the class is specific for the Standard Workload Format (*.swf)
  from The Hebrew University of Jerusalem.

- Introduces the GoogleMachineEventsTraceReader to create Hosts
  from Google Cloud trace files.

  * Adds startTime and shutdownTime attributes to Host
  * Enables to request the creation of Hosts only at the time
    specified by the timestamp in trace file.
  * Enables processing of ADD and REMOVE event types.
  * Adds GoogleMachineEventsExample1.

- Introduces the GoogleTaskEventsTraceReader
  to read tasks events from Google Cloud traces.
  Jobs files are not read because they don't have useful information
  for simulations and are represented in CloudSim Plus just as an integer
  value for the jobId attribute of a Cloudlet.

  * Adds new Cloudlet status to conform to the Google Cluster Trace tasks events
  * Adds new CloudSimTags to enable a broker to receive Cloudlet status changes
    when creating Cloudlets from a trace file such as the Google Cluster Trace
  * Adds a jobId attribute to Cloudlet so that Cloudlets
    can be categorized as belonging to a fictitious job.
  * Adds DatacenterBroker.getCloudletSubmittedList
  * Adds GoogleTaskEventsExample1.

- FieldIndex enums in GoogleTraceReaderAbstract subclasses
  enable getting a value from the parsed trace line directly
  from a method inside the enum. This way, the value is returned
  in the correct type and after making unit conversions
  (if required). That was accomplished by making the enum to
  implement the TraceField interface.

- Adds google-cluster-data-samples.xlsx
  as an easier way to analyse the structure of Google Trace files.
- Adds sample google trace files to the resource directory
  of the examples project.

- Fix Conversion issues (such as GB to MB) and updates documentation.
- Changes stream and "enhanced for" in CloudSim class to iterators
  and indexed loops to improve efficiency and avoid
  ConcurrentModificationException.
  Before it was being created defensive copies of event and entities list
  to avoid such an exception, but reducing performance when
  there is a large number of objects in such lists.

Signed-off-by: Manoel Campos <manoelcampos@gmail.com>

manoelcampos added a commit that referenced this issue Aug 10, 2018

Closes #149 :: Process trace files from Google Cluster Data
- Refactors WorkloadFileReader extracting methods to a superclasse to enable
  creating new subclasses for different trace formats
  (such as Google Cluster Data)
- Renames WorkloadFileReader to SwfWorkloadFileReader
  because the class is specific for the Standard Workload Format (*.swf)
  from The Hebrew University of Jerusalem.

- Introduces the GoogleMachineEventsTraceReader to create Hosts
  from Google Cloud trace files.

  * Adds startTime and shutdownTime attributes to Host
  * Enables to request the creation of Hosts only at the time
    specified by the timestamp in trace file.
  * Enables processing of ADD and REMOVE event types.
  * Adds GoogleMachineEventsExample1.

- Introduces the GoogleTaskEventsTraceReader
  to read tasks events from Google Cloud traces.
  Jobs files are not read because they don't have useful information
  for simulations and are represented in CloudSim Plus just as an integer
  value for the jobId attribute of a Cloudlet.

  * Adds new Cloudlet status to conform to the Google Cluster Trace tasks events
  * Adds new CloudSimTags to enable a broker to receive Cloudlet status changes
    when creating Cloudlets from a trace file such as the Google Cluster Trace
  * Adds a jobId attribute to Cloudlet so that Cloudlets
    can be categorized as belonging to a fictitious job.
  * Adds DatacenterBroker.getCloudletSubmittedList
  * Adds GoogleTaskEventsExample1.

- FieldIndex enums in GoogleTraceReaderAbstract subclasses
  enable getting a value from the parsed trace line directly
  from a method inside the enum. This way, the value is returned
  in the correct type and after making unit conversions
  (if required). That was accomplished by making the enum to
  implement the TraceField interface.

- Adds google-cluster-data-samples.xlsx
  as an easier way to analyse the structure of Google Trace files.
- Adds sample google trace files to the resource directory
  of the examples project.

- Fix Conversion issues (such as GB to MB) and updates documentation.
- Changes stream and "enhanced for" in CloudSim class to iterators
  and indexed loops to improve efficiency and avoid
  ConcurrentModificationException.
  Before it was being created defensive copies of event and entities list
  to avoid such an exception, but reducing performance when
  there is a large number of objects in such lists.

Signed-off-by: Manoel Campos <manoelcampos@gmail.com>

manoelcampos added a commit that referenced this issue Aug 10, 2018

Closes #149 :: Process trace files from Google Cluster Data
- Refactors WorkloadFileReader extracting methods to a superclasse to enable
  creating new subclasses for different trace formats
  (such as Google Cluster Data)
- Renames WorkloadFileReader to SwfWorkloadFileReader
  because the class is specific for the Standard Workload Format (*.swf)
  from The Hebrew University of Jerusalem.

- Introduces the GoogleMachineEventsTraceReader to create Hosts
  from Google Cloud trace files.

  * Adds startTime and shutdownTime attributes to Host
  * Enables to request the creation of Hosts only at the time
    specified by the timestamp in trace file.
  * Enables processing of ADD and REMOVE event types.
  * Adds GoogleMachineEventsExample1.

- Introduces the GoogleTaskEventsTraceReader
  to read tasks events from Google Cloud traces.
  Jobs files are not read because they don't have useful information
  for simulations and are represented in CloudSim Plus just as an integer
  value for the jobId attribute of a Cloudlet.

  * Adds new Cloudlet status to conform to the Google Cluster Trace tasks events
  * Adds new CloudSimTags to enable a broker to receive Cloudlet status changes
    when creating Cloudlets from a trace file such as the Google Cluster Trace
  * Adds a jobId attribute to Cloudlet so that Cloudlets
    can be categorized as belonging to a fictitious job.
  * Adds DatacenterBroker.getCloudletSubmittedList
  * Adds GoogleTaskEventsExample1.

- FieldIndex enums in GoogleTraceReaderAbstract subclasses
  enable getting a value from the parsed trace line directly
  from a method inside the enum. This way, the value is returned
  in the correct type and after making unit conversions
  (if required). That was accomplished by making the enum to
  implement the TraceField interface.

- Adds google-cluster-data-samples.xlsx
  as an easier way to analyse the structure of Google Trace files.
- Adds sample google trace files to the resource directory
  of the examples project.

- Fix Conversion issues (such as GB to MB) and updates documentation.
- Changes stream and "enhanced for" in CloudSim class to iterators
  and indexed loops to improve efficiency and avoid
  ConcurrentModificationException.
  Before it was being created defensive copies of event and entities list
  to avoid such an exception, but reducing performance when
  there is a large number of objects in such lists.

Signed-off-by: Manoel Campos <manoelcampos@gmail.com>

manoelcampos added a commit that referenced this issue Aug 10, 2018

Closes #149 :: Process trace files from Google Cluster Data
- Refactors WorkloadFileReader extracting methods to a superclasse to enable
  creating new subclasses for different trace formats
  (such as Google Cluster Data)
- Renames WorkloadFileReader to SwfWorkloadFileReader
  because the class is specific for the Standard Workload Format (*.swf)
  from The Hebrew University of Jerusalem.

- Introduces the GoogleMachineEventsTraceReader to create Hosts
  from Google Cloud trace files.

  * Adds startTime and shutdownTime attributes to Host
  * Enables to request the creation of Hosts only at the time
    specified by the timestamp in trace file.
  * Enables processing of ADD and REMOVE event types.
  * Adds GoogleMachineEventsExample1.

- Introduces the GoogleTaskEventsTraceReader
  to read tasks events from Google Cloud traces.
  Jobs files are not read because they don't have useful information
  for simulations and are represented in CloudSim Plus just as an integer
  value for the jobId attribute of a Cloudlet.

  * Adds new Cloudlet status to conform to the Google Cluster Trace tasks events
  * Adds new CloudSimTags to enable a broker to receive Cloudlet status changes
    when creating Cloudlets from a trace file such as the Google Cluster Trace
  * Adds a jobId attribute to Cloudlet so that Cloudlets
    can be categorized as belonging to a fictitious job.
  * Adds DatacenterBroker.getCloudletSubmittedList
  * Adds GoogleTaskEventsExample1.

- FieldIndex enums in GoogleTraceReaderAbstract subclasses
  enable getting a value from the parsed trace line directly
  from a method inside the enum. This way, the value is returned
  in the correct type and after making unit conversions
  (if required). That was accomplished by making the enum to
  implement the TraceField interface.

- Adds google-cluster-data-samples.xlsx
  as an easier way to analyse the structure of Google Trace files.
- Adds sample google trace files to the resource directory
  of the examples project.

- Fix Conversion issues (such as GB to MB) and updates documentation.
- Changes stream and "enhanced for" in CloudSim class to iterators
  and indexed loops to improve efficiency and avoid
  ConcurrentModificationException.
  Before it was being created defensive copies of event and entities list
  to avoid such an exception, but reducing performance when
  there is a large number of objects in such lists.

Signed-off-by: Manoel Campos <manoelcampos@gmail.com>

manoelcampos added a commit that referenced this issue Aug 10, 2018

Closes #149 :: Process trace files from Google Cluster Data
- Refactors WorkloadFileReader extracting methods to a superclasse to enable
  creating new subclasses for different trace formats
  (such as Google Cluster Data)
- Renames WorkloadFileReader to SwfWorkloadFileReader
  because the class is specific for the Standard Workload Format (*.swf)
  from The Hebrew University of Jerusalem.

- Introduces the GoogleMachineEventsTraceReader to create Hosts
  from Google Cloud trace files.

  * Adds startTime and shutdownTime attributes to Host
  * Enables to request the creation of Hosts only at the time
    specified by the timestamp in trace file.
  * Enables processing of ADD and REMOVE event types.
  * Adds GoogleMachineEventsExample1.

- Introduces the GoogleTaskEventsTraceReader
  to read tasks events from Google Cloud traces.
  Jobs files are not read because they don't have useful information
  for simulations and are represented in CloudSim Plus just as an integer
  value for the jobId attribute of a Cloudlet.

  * Adds new Cloudlet status to conform to the Google Cluster Trace tasks events
  * Adds new CloudSimTags to enable a broker to receive Cloudlet status changes
    when creating Cloudlets from a trace file such as the Google Cluster Trace
  * Adds a jobId attribute to Cloudlet so that Cloudlets
    can be categorized as belonging to a fictitious job.
  * Adds DatacenterBroker.getCloudletSubmittedList
  * Adds GoogleTaskEventsExample1.

- FieldIndex enums in GoogleTraceReaderAbstract subclasses
  enable getting a value from the parsed trace line directly
  from a method inside the enum. This way, the value is returned
  in the correct type and after making unit conversions
  (if required). That was accomplished by making the enum to
  implement the TraceField interface.

- Adds google-cluster-data-samples.xlsx
  as an easier way to analyse the structure of Google Trace files.
- Adds sample google trace files to the resource directory
  of the examples project.

- Fix Conversion issues (such as GB to MB) and updates documentation.
- Changes stream and "enhanced for" in CloudSim class to iterators
  and indexed loops to improve efficiency and avoid
  ConcurrentModificationException.
  Before it was being created defensive copies of event and entities list
  to avoid such an exception, but reducing performance when
  there is a large number of objects in such lists.

Signed-off-by: Manoel Campos <manoelcampos@gmail.com>

manoelcampos added a commit that referenced this issue Aug 17, 2018

Closes #149 :: Process trace files from Google Cluster Data
- Refactors WorkloadFileReader extracting methods to a superclasse to enable
  creating new subclasses for different trace formats
  (such as Google Cluster Data)
- Renames WorkloadFileReader to SwfWorkloadFileReader
  because the class is specific for the Standard Workload Format (*.swf)
  from The Hebrew University of Jerusalem.

- Introduces the GoogleMachineEventsTraceReader to create Hosts
  from Google Cloud trace files.

  * Adds startTime and shutdownTime attributes to Host
  * Enables to request the creation of Hosts only at the time
    specified by the timestamp in trace file.
  * Enables processing of ADD and REMOVE event types.
  * Adds GoogleMachineEventsExample1.

- Introduces the GoogleTaskEventsTraceReader
  to read tasks events from Google Cloud traces.
  Jobs files are not read because they don't have useful information
  for simulations and are represented in CloudSim Plus just as an integer
  value for the jobId attribute of a Cloudlet.

  * Adds new Cloudlet status to conform to the Google Cluster Trace tasks events
  * Adds new CloudSimTags to enable a broker to receive Cloudlet status changes
    when creating Cloudlets from a trace file such as the Google Cluster Trace
  * Adds a jobId attribute to Cloudlet so that Cloudlets
    can be categorized as belonging to a fictitious job.
  * Adds DatacenterBroker.getCloudletSubmittedList
  * Adds GoogleTaskEventsExample1.
  * Updates Cloudlets attributes during simulation time,
    according to the values read from the "task events" trace file.

- FieldIndex enums in GoogleTraceReaderAbstract subclasses
  enable getting a value from the parsed trace line directly
  from a method inside the enum. This way, the value is returned
  in the correct type and after making unit conversions
  (if required). That was accomplished by making the enum to
  implement the TraceField interface.

- Adds google-cluster-data-samples.xlsx
  as an easier way to analyse the structure of Google Trace files.
- Adds sample google trace files to the resource directory
  of the examples project.

- Fix Conversion issues (such as GB to MB) and updates documentation.
- Changes stream and "enhanced for" in CloudSim class to iterators
  and indexed loops to improve efficiency and avoid
  ConcurrentModificationException.
  Before it was being created defensive copies of event and entities list
  to avoid such an exception, but reducing performance when
  there is a large number of objects in such lists.

Signed-off-by: Manoel Campos <manoelcampos@gmail.com>

manoelcampos added a commit that referenced this issue Aug 28, 2018

Closes #149 :: Process trace files from Google Cluster Data
- Refactors WorkloadFileReader by extracting methods to a superclass to enable
  creating new subclasses for different trace formats
  (such as Google Cluster Data).
- Renames WorkloadFileReader to SwfWorkloadFileReader
  because the class is specific for the Standard Workload Format (*.swf)
  from The Hebrew University of Jerusalem.

- Introduces the GoogleMachineEventsTraceReader class to create Hosts
  from Google Cloud trace files:

  * Adds startTime and shutdownTime attributes to Host;
  * Enables requesting the creation of Hosts only at the time
    specified by the timestamp in trace file;
  * Enables processing of ADD and REMOVE event types;
  * Adds GoogleMachineEventsExample1.

- Introduces the GoogleTaskEventsTraceReader
  to read tasks events from Google Cloud traces.
  Jobs files are not read because they don't have useful information
  for simulations. They are represented in CloudSim Plus just as integer
  values for the jobId attribute of a Cloudlet:

  * Adds new Cloudlet status to conform to the Google Cluster Trace tasks events;
  * Adds new CloudSimTags to enable a broker to receive Cloudlet status changes
    when creating Cloudlets from a trace file;
  * Adds a jobId attribute to Cloudlet so that it
    can be categorized as belonging to a fictitious job;
  * Adds DatacenterBroker.getCloudletSubmittedList;
  * Adds GoogleTaskEventsExample1;
  * Updates Cloudlet's attributes during simulation time,
    according to the values read from the "task events" trace file
    (such as max number of CPU cores and max RAM usage).

- FieldIndex enums in GoogleTraceReaderAbstract subclasses
  enable getting a value from the parsed trace line directly
  from a method inside the enum. This way, the value is returned
  in the correct type and after making unit conversions
  (if required). That was accomplished by making the enum to
  implement the TraceField interface.

- Adds google-cluster-data-samples.xlsx
  as an easier way to analyse the structure of Google Trace files.
- Adds sample google trace files to the resource directory
  of the examples project.

- Fix Conversion issues (such as GB to MB) and updates documentation.
- Changes stream and "enhanced for" in CloudSim class to iterators
  and indexed loops to improve efficiency and avoid
  ConcurrentModificationException.
  Previously it was being created defensive copies of events and entities lists
  to avoid such an exception, but reducing performance when
  there is a large number of objects in such lists.

- Changes the type of the CloudSimEvent's data attribute to
  a Runnable when the event tag is CloudSimTags.CLOUDLET_UPDATE_ATTRIBUTES.
  This way, all the logic to update the Cloudlet's attributes
  can be customized by the researcher and encapsulated
  into a no-args and no-return function (the Runnable).
  This function is then executed by the broker when
  an event of such a type is received.
- Defines the correct way to interpret the "resource request" fields
  in the "task events" trace file:
  * The "resource request for CPU" is used to define the Cloudlet's PEs
  * The "resource request for RAM" is used to define max resource utilization
    for the Ram UtilizationModel of a Cloudlet.
    In this case, it's required a UtilizationModelDynamic instance.

- Introduces the GoogleTaskUsageTracerReader class to read "task usage" trace files.

Signed-off-by: Manoel Campos <manoelcampos@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment