Skip to content

Review of Guideline #2

Jump to bottom
philippemilletresearch edited this page Jul 19, 2018 · 38 revisions

Review of Guideline #2

Formatting

Item Outcome
Guideline complies with the Guideline-Template No, but I updated it!, so... Yes!
Name of guideline responsible with affiliation is clearly stated Yes

Is the audience of the guideline correctly specified?

I would rather think that, since this guideline goal is the selection of an operating system, the audience should be only "System architects". "OS designers" should not have to follow a selection methodology because they do produce an OS, so they don't have to select anything.

Is the expertise required to write the guideline correctly specified?

YES!

To produce a methodology for the selection of the right operating system, I think one need to understand how an operating system is made and what are the differences between the big list of available OSes. This can only be done if you tried once to implement a real-time operating system or if you already used several of them. I recommend to keep "OS designers:" expertise for this guideline.

Are the keywords well selected?

No.

It was Choice of (Real-Time) Operating System, and I changed it into Component selection to have a wider spectrum of topics attached to the keyword.

Are parts of the guideline too generic or too specific?

  1. The guideline is a bit too generic. It does not help me choosing the OS. It explains a bit what should the requirements be in order to select an OS, but it keeps the reader in expectation of a real method to select the OS. E.g. why would I choose HipperOS (link) rather than FreeRTOS (link)? or FastOS (link)? or PikeOS (link)? or Integrity (link)? or Zephyr (link)?

    A list of RTOS is available on Wikipedia (here). This list can be used as a bootstrap for our own list of OS selection criteria.

  2. There must be a clear view of the choices for the users. What OS they can use? What are the benefit from each of them?

    Some information can be used or extracted from the Real-Time Operating system page on Wikipedia (here).

    • The guideline should exhibit a methodology that helps in the OS selection.

    • The application domain should be taken into account while it will require libraries that might be already ported to a given operating system.

    • The type of supported hardware is very important too.

    • The ease of porting from Linux to the operating system (i.e. the OS APIs) must also be explained.

    • The memory footprints: this is linked with the supported hardware. Smaller memory size restricts the number of OS one can use.

    • The latency: this is linked with the application. The capability to guaranty latencies and/or real-time deadlines can be critical for some applications, e.g. in real-time medical imaging.

    • European Solution: This is also to be taken into consideration because it can allow faster interaction with European partners but also it removes ITAR or other American controls.

    • Price & License: is often a important factor together with the license because it can impact what the customer can do with the operating system and impacts the final production costs. One selection criteria must be the availability of evaluation licenses.

    • Support: for critical applications and industrial applications the support from the vendor is crucial. Or at least there must be an expert somewhere that can answer the questions. One way is to have a in-house expert that will spend time, take the knowledge and understand the OS as if he created it. Another way is to have support, either from an ecosystem or from a company that sells the OS.

  3. Figures must be given in the deliverable to characterize the available operating systems. The guideline should describe how to use the figures in order to select the OS. I feel confident that HipperOS has that kind of information because it is critical for the company to be able to convince customers that the OS is better than the others.

  4. The recommendations are too generic. Concrete examples should be given to clearly explain what to do. For instance in the second paragraph: If your design needs a high degree of reliability, what is meant with high degree of reliability? If relevant, one could refer to safety integrity level (SIL) (link), avionic software DO-178B (link), medical software (link.1, link.2, link.3), and/or explain how to determine the degree of reliability. In the guideline text it is not clear if we really mean reliability or safety.

Does the guideline explicitly refer to the handbook? To which part of the deliverable is it relevant (e.g., chapter of D1.2/D1.3)?

The guideline definitely refers to the chapter 4 of D1.2 & D1.3.

  1. Both the chapter and the guideline must be extended. Refer to the comments above.
  2. The book must include the list of the main competitor OS. A short list is given in this review. It might be updated.

Proposed methodology:

  1. make a clear list of the requirements (we can help the user here by giving the requirement list we used in Tulipp)
  2. track the requirements down to operating system specification (we can also give the list of the specification we used in Tulipp)
  3. compare the specifications list with the specification of the available OSes (the list of OSes is given in the book along with the requirements => refer to the book)
  4. evaluate the specifications that are required for the application and not known from the OSes. At this stage, we will not evaluate ALL the possible OS. We should have selected one or two of them. Then we ask for evaluation licenses.
  5. Implement the critical parts of the application to validate the OS choice.

Does the guideline specify the work done in the project that can benefit from the guideline?

YES!

This guideline is a keystone of the project because it explains how to select a crucial component: the operating system. Because of this crucial status, it must be updated together with the handbook to give the user a consistent information. The methodology should be explained in the book. The guideline should explain how to use the methodology with examples, following that methodology with one or more of the Tulipp use cases.

Other comments

  1. Guideline advice: The advice should be short and would be to follow the methodology proposed in our handbook in order to select the RTOS that best fits the application needs.

  2. Insights that led to the guideline: The current insight: This recommendation is based on theoretical and experimental background information regarding real-time systems for critical applications.

    Is not actually an insight... I propose the following:

    Former embedded application where developed "bare metal", i.e. directly on the hard. While this allows to directly access the hardware, it does not abstract it which lead to the need to completely redo the application when changing the hardware. With time, libraries can be defined with the most used elements but it is still difficult to manage their consistency. Operating systems are more and more used in the embedded application domain while it allows to abstract the hardware and capitalize from the number of application using it, which in turn makes it more and more reliable. There are a lot of available operating systems and one can even still choose to develop the application bare metal. However when starting the development of an embedding product, one has to select the right operating system which is a difficult task while the differences between them, the key selection factors are often not clearly exhibited. Therefore one need key information on the available operating systems and a methodology to select the right one.

  3. Recommended implementation method of the guideline along with a solid motivation for the recommendation: This is where we explain the methodology. This methodology will also be given in the handbook. We can have a shorter version here and refer to the book for the full version.

  4. Instantiation of the recommended implementation method in the reference platform: This is where we use the methodology to show how it is used and we end up selecting HipperOS. Here we must take into account the application domain: image processing + Real-time + some human-life at stake. There must be some figures to make the selection concrete and scientific. This part will not be in the book and only in the guideline.

  5. Evaluation of the guideline in reference applications: This is where we must take into account the Tulipp use cases. Ideally, each use case should evaluate the choice. Does it fulfill all the requirements? Each time a use case is implemented on top of HipperOS, it is evaluated. As long as the implemented UC is fulfilling the requirements the evaluation is positive. In case something does not meet the requirement, we must investigate the reason and explain what we can do to workaround or what should be changed (in the reference platform, the guideline, the application...).

  6. References: There are not reference in the guideline. Obvious references must be done to the deliverables, handbook. The website of the operating systems must also be given.

Track changes:

  1. 18/07/2018: guideline reviewed. Made some formatting changes in the guidelines to cope with template and update the keywords and categories.

TULIPP Guideline Wiki

Clone this wiki locally