Fix tasks states #301

derkling · 2017-02-17T11:37:40Z

No description provided.

bjackman

I seem to be in a nitpicky mood today but looks useful.

bjackman · 2017-02-17T12:15:13Z

libs/utils/env.py

@@ -644,6 +644,17 @@ def _init_platform(self):
        # Adding topology information
        self.platform['topology'] = self.topology.get_level("cluster")

+        # Adding kernel build information
+        kver = self.target.execute('uname -r').strip()


Maybe we should extend devlib's KernelVersion instead of having the regex and uname call in here? I don't mind though, up to you.

Again I don't want to possibly break some devlib APIs... however, here the main goal for the regexp is to report the major.minor version and the exact SHA1 within the platform.json.

Otherwise we should update devlib to build a namedtuple with all these values...

No need to change any existing APIs, the only function in KernelVersion right now is __str__/__repr__ so we could just add properties.

Added this new interface in:
ARM-software/devlib#90

... let's wait for this to be merged and than I'll add a refresh of devlib to this PR.

bjackman · 2017-02-17T12:23:54Z

libs/utils/env.py

+            self.platform['kernel_version'] = match.group('ver')
+            self.platform['kernel_sha1'] = match.group('sha1')
+        self.platform['kernel_version_full'] = kver
+        self.platform['build'] = str(self.target.kernel_version)


I suggest either putting all the kernel version stuff under a sub-dict keyed with "kernel" (self.platform['kernel'] = {'build': foo, 'sha1': bar, ...}) or changing '"build"' to '"kernel_build"'.

That make sense... again perhaps better a namedtuple returned by devlib...

bjackman · 2017-02-17T12:27:25Z

libs/utils/analysis/latency_analysis.py

+        kver = float(self._trace.platform.get('kernel_version', '3.18'))
+        if kver > 4.7:
+            TASK_STATES[2048] = "n" # TASK_NEW
+        TASK_MAX_STATE = 2*sorted(TASK_STATES.keys())[-1]


sorted(x)[-1] can just be max(x)

bjackman · 2017-02-17T12:40:42Z

libs/utils/env.py

+        kver = self.target.execute('uname -r').strip()
+        match = KVERSION_RE.match(kver)
+        if match:
+            self.platform['kernel_version'] = match.group('ver')


Might be handy to have this as a tuple of ints (like (3, 18) instead of "3.18"), then we would leave space for richer info (e.g. stable kernels like 4.9.10) in the future. This works nicely because tuples sort lexicographically, so in the later commit you could have if self.platform.get('kernel_version', (0,)) > (4, 7):

I'm using a dict because I need to dump it as a JSON... the new KernelVersion set of attributes reports separate attributes for version.major.minor... we can use these attributes to build whatever tuple we need at client side.

derkling · 2017-02-20T15:35:41Z

I guess that, as long as ARM-software/devlib#90 gets merged, this PR can enter as well.

bjackman · 2017-02-20T16:34:44Z

libs/utils/env.py

+        kver = self.target.kernel_version
+        self.platform['kernel'] = {t: getattr(kver, t, '') \
+            for t in [
+                'release', 'version',


KernelVersion doesn't have release or version attributes

They are actually there:
https://github.com/ARM-software/devlib/blob/master/devlib/target.py#L1196

Oh yeah, sorry!

bjackman · 2017-02-20T16:35:09Z

libs/utils/env.py

@@ -707,6 +707,18 @@ def _init_platform(self):
        # Adding topology information
        self.platform['topology'] = self.topology.get_level("cluster")

+        # Adding kernel build information
+        kver = self.target.kernel_version
+        self.platform['kernel'] = {t: getattr(kver, t, '') \


Nit: don't need the \

derkling · 2017-02-20T18:17:31Z

libs/utils/env.py

@@ -977,4 +989,8 @@ def _feature(self, feature):
    r'Bcast:(.*) '
 )

+KVERSION_RE = re.compile(


Now that we have the code in devlib... this has to be removed! :-/

derkling · 2017-02-20T18:18:47Z

libs/utils/analysis/latency_analysis.py

        }
-        TASK_MAX_STATE = 4096
+        kver = float(self._trace.platform.get('kernel_version', '3.18'))
+        if kver > 4.7:


If @bjackman adds the new cmp API to devlib.... this should be updated as well...

I didn't add the __cmp__ in the end. However this needs to change anyway because 'kernel_version' isn't a field any more. I put a couple of suggested fixup patches here: https://github.com/bjackman/lisa/commits/fix-tasks-states

These platform data can be useful to know exactly on which platform configuration tests have been executed. Specifically, the "kernel_version" and "kernel_sha1" can be useful to cross-check a kernel with the expected results. Moreover, some of these information (e.g. kernel_version) is useful to tune some analysis methods depending on expected kernel feature (e.g. available tracepoint, constants, etc.) Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>

This column is not currently required by the latency analysis moreover this field is not more produced since kernel v4.4. Let's disregard it. Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>

Certain tasks flags are available only on more recent kernel versions. Let's use the platform reported kernel_version number to add them when the trace support them. Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>

jlelli · 2017-02-27T14:28:58Z

Looks good to me. I've also indirectly tested it by using the new latency analysis API.

f7a3ecbbf Merge pull request ARM-software#302 from valschneider/ftrace_function 2ea55a58e ftrace: Add support to parse function tracing ('function' tracer) 412924f28 Merge pull request ARM-software#301 from JaviMerino/fix_documentation e231aca72 Merge pull request ARM-software#300 from douglas-raillard-arm/fix_custom_scope b25800328 doc: Update Dynamic traces 5babb40b6 ftrace: Add endtime attribute 23a35ecae ftrace: Avoid storing trace-cmd report output in memory 9d67e5555 tests: Fix unit tests 08c7da476 ftrace: Only add dynamic classes on non-custom scopes git-subtree-dir: external/trappy git-subtree-split: f7a3ecbbfb4a92031431dca31c48eb67ac9db08d

derkling assigned jlelli Feb 17, 2017

derkling requested a review from jlelli February 17, 2017 11:37

bjackman reviewed Feb 17, 2017

View reviewed changes

derkling mentioned this pull request Feb 20, 2017

Add profiling analysis #302

Merged

derkling added this to the 17.02 milestone Feb 20, 2017

derkling force-pushed the fix-tasks-states branch 3 times, most recently from 1e34d90 to 7bc91d3 Compare February 20, 2017 16:14

bjackman suggested changes Feb 20, 2017

View reviewed changes

derkling force-pushed the fix-tasks-states branch from 7bc91d3 to 04663d2 Compare February 20, 2017 17:46

derkling commented Feb 20, 2017

View reviewed changes

derkling force-pushed the fix-tasks-states branch from 04663d2 to 97c7bb4 Compare February 22, 2017 11:50

derkling added 3 commits February 27, 2017 14:11

utils/analysis/latency: remove unused columns

0f9f617

This column is not currently required by the latency analysis moreover this field is not more produced since kernel v4.4. Let's disregard it. Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>

utils/analysis/latency: make tasks tagging more robust

ebc066e

Certain tasks flags are available only on more recent kernel versions. Let's use the platform reported kernel_version number to add them when the trace support them. Signed-off-by: Patrick Bellasi <patrick.bellasi@arm.com>

derkling force-pushed the fix-tasks-states branch from 97c7bb4 to ebc066e Compare February 27, 2017 14:11

jlelli merged commit 955bab2 into ARM-software:master Feb 27, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix tasks states #301

Fix tasks states #301

derkling commented Feb 17, 2017

bjackman left a comment

bjackman Feb 17, 2017

derkling Feb 17, 2017

bjackman Feb 17, 2017

derkling Feb 20, 2017

bjackman Feb 17, 2017

derkling Feb 17, 2017

derkling Feb 20, 2017

bjackman Feb 17, 2017

derkling Feb 20, 2017

bjackman Feb 17, 2017

derkling Feb 20, 2017

derkling commented Feb 20, 2017

bjackman Feb 20, 2017

derkling Feb 20, 2017

bjackman Feb 20, 2017

bjackman Feb 20, 2017

derkling Feb 20, 2017

derkling Feb 20, 2017

bjackman Feb 20, 2017

jlelli commented Feb 27, 2017

Fix tasks states #301

Fix tasks states #301

Conversation

derkling commented Feb 17, 2017

bjackman left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

derkling commented Feb 20, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jlelli commented Feb 27, 2017