Add pretty bars for compile output #5929

geky · 2018-01-25T00:44:09Z

Looks like this:

$ mbed compile -m ARCH_PRO -t GCC_ARM
Building project mbed-os-prettyoutput (ARCH_PRO, GCC_ARM)
Scan: .
Scan: env
Scan: mbed
Scan: FEATURE_LWIP
Text 70.5KB Data 2.72KB BSS 7.43KB                 ROM 73.2KB RAM 10.1KB
ROM [|||||||                                             ]  73.2KB/512KB
RAM [||||||||||||||||                                    ]   10.1KB/32KB
Image: BUILD/ARCH_PRO/GCC_ARM/mbed-os-prettyoutput.bin

This is an alternative to the pretty table output that provides useful information about memory consumption without using as much screen real-estate as the pretty table output. You may also note this matches what is displayed on the website, which is an added plus.

cc @theotherjimmy, @0xc0170

kegilbert · 2018-01-25T00:50:41Z

tools/memap.py

+        Returns: string containing nice looking bars
+        """
+
+        # TODO add tty detection, and width detection probably


Was this a TODO you wanted to address before merging or is it fine as is?

Either or. You can see the current width detection below, which will only work on Linux (but doesn't hurt other OSs):

mbed-os/tools/memap.py

Lines 628 to 638 in a4bfd01

# TODO add tty detection, and width detection probably

WIDTH = 72

try:

# NOTE this only works on linux

import sys, fcntl, termios, struct

height, width, _, _ = struct.unpack('HHHH',

fcntl.ioctl(sys.stdout.fileno(), termios.TIOCGWINSZ,

struct.pack('HHHH', 0, 0, 0, 0)))

WIDTH = min(width, WIDTH)

except:

pass

I'm not planning on adding support for other OSs at this time, though I don't think that should stop this PR, they're just stuck at a 72 character width. I did search briefly, but unfortunately finding cross-platform screen width is a nightmare in python 2.7.

Fair enough.

kegilbert · 2018-01-25T00:53:30Z

I like it!

Sorry if I misundertood, but is there a compile flag to allow a user to specify which format they'd like (table versus graph)? Looks like the table print was removed completely.

geky · 2018-01-25T01:07:24Z

Updated 👍

I repurposed --stats-depth to revert back to pretty table printing. So no --stats-depth = pretty bars.

drewcassidy · 2018-01-25T01:51:22Z

this might be out of scope but there are unicode characters for doing progress bars at different subdivisions of one character, U+2588 █ through U+258F ▏

0xc0170 · 2018-01-25T08:47:12Z

Looks neat 👍

This is using cmsis packs - what happens if device is not yet in the index ? From the code looks like it would fail? As I recall they are used for bootloader or exporters, with this change would be also for compilation. In case of exporter, if device is not yet in the packs, it fails for uvision to export. What is the case for compilation with this patch?

geky · 2018-01-25T16:57:58Z

@drewcassidy, Oh that's a good idea. Although it would look bad on non-unicode terminals. Do you know if there's a way to detect unicode support from a terminal? Do you think we can rely on $LANG?

@0xc0170, good point, I'll wrap all of this in a try catch based on cmsis-pack availability.

drewcassidy · 2018-01-25T17:07:04Z

Do you think we can rely on $LANG?

$LANG appears to work, at least in iterm and terminal on macOS. You could only enable it if $LANG explicitly mentions UTF-8, and/or have a global flag to disable it like some tools have for color support. I was thinking of contributing to prettytable to add unicode block drawing support, so the flag could be used for configuring that as well

geky · 2018-01-25T17:07:08Z

@0xc0170, updated, @theotherjimmy suggested I can rely on the presence of device_name in the target object to indicate if cmsis-packs are available. CI can now verify that claim :)

Here's the output without a cmsis-pack, though I'm open to suggestions if we want to see something different:

$ mbed compile -m ARM_BEETLE_SOC -t GCC_ARM
Building project mbed-os-prettyoutput (ARM_BEETLE_SOC, GCC_ARM)
Scan: .                                                        
Scan: env                                                      
Scan: mbed                                                     
Scan: FEATURE_BLE                                              
Text 99KB Data 2.84KB BSS 13KB ROM 102KB RAM 15.8KB            
Image: BUILD/ARM_BEETLE_SOC/GCC_ARM/mbed-os-prettyoutput.bin

geky · 2018-01-25T17:21:38Z

@drewcassidy, I think the unicode printing is a neat idea, but I don't think I'm going to be able to implement it at the moment. Feel free to add it in or include it when you add unicode support to the pretty tables.

theotherjimmy · 2018-01-25T17:23:41Z

@geky Good pick of platform! that one probably won't ever have a CMSIS Pack. That output looks great! I'll review now.

geky · 2018-01-25T17:25:50Z

@theotherjimmy, I just chose the first platform that CI failed on. Speaking of CI failures, apparently the NUCLEO_F767ZI doesn't have any ram? Updated with another catch on cmsis-pack parsing errors.

theotherjimmy · 2018-01-25T17:25:26Z

tools/build_api.py

-            memap_table = memap_instance.generate_output('table', stats_depth)
-
+            real_stats_depth = stats_depth if stats_depth is None else 2
+            memap_table = memap_instance.generate_output('table', real_stats_depth)


Could you hove these lines into the else statement below? that way we only generate what we need.

memap_table is also used by the build report below, and the generate_output has side-effects I'm not sure about. You can see it was originally outside of the if not silent statement.

memap_table is also used by the build report below

That's a good point.

generate_output has side-effects I'm not sure about

It does a compute_report followed by a reduce_depth, which will be run by the following generate_output calls anyway.

theotherjimmy · 2018-01-25T17:26:58Z

tools/memap.py

+                    struct.pack('HHHH', 0, 0, 0, 0)))
+            WIDTH = min(width, WIDTH)
+        except:
+            pass


Could you assign width to 72 here instead? and make that except not catch C-c and similar with

except Exception: WIDTH = 72

WIDTH is also used on line 636 (right above). Will update the second part of your sentence.

theotherjimmy · 2018-01-25T17:29:27Z

tools/memap.py

+
+        text = sum(module['size']['.text']
+            for module in self.mem_report
+            if 'size' in module)


Could this be:

text = self.subtotal['.text']

theotherjimmy · 2018-01-25T17:29:41Z

tools/memap.py

+            if 'size' in module)
+        data = sum(module['size']['.data']
+            for module in self.mem_report
+            if 'size' in module)


Could this be:

data = self.subtotal['.data']

theotherjimmy · 2018-01-25T17:30:12Z

tools/memap.py

+            if 'size' in module)
+        bss = sum(module['size']['.bss']
+            for module in self.mem_report
+            if 'size' in module)


Could this be:

bss = self.subtotal['.bss']

theotherjimmy · 2018-01-25T17:30:39Z

tools/memap.py

+        bss = sum(module['size']['.bss']
+            for module in self.mem_report
+            if 'size' in module)
+        rom_used = text + data


Could this be:

rom_used = self.mem_summary['total_flash']

Why not just add them together? The math below would have problems if they differed

theotherjimmy · 2018-01-25T17:31:15Z

tools/memap.py

+            for module in self.mem_report
+            if 'size' in module)
+        rom_used = text + data
+        ram_used = data + bss


Could this be:

ram_used = self.mem_summary['static_ram']

theotherjimmy · 2018-01-25T17:33:54Z

@geky Who knew? F767ZI has no RAM, huh.

Looks like this: Building project mbed-os-prettyoutput (ARCH_PRO, GCC_ARM) Scan: . Scan: env Scan: mbed Scan: FEATURE_LWIP Text 70.5KB Data 2.72KB BSS 7.43KB ROM 73.2KB RAM 10.1KB ROM [||||||| ] 73.2KB/512KB RAM [|||||||||||||||| ] 10.1KB/32KB Image: BUILD/ARCH_PRO/GCC_ARM/mbed-os-prettyoutput.bin If you build a target without a cmsis-pack it looks like this: Building project mbed-os-prettyoutput (ARM_BEETLE_SOC, GCC_ARM) Scan: . Scan: env Scan: mbed Scan: FEATURE_BLE Text 99KB Data 2.84KB BSS 13KB ROM 102KB RAM 15.8KB Image: BUILD/ARM_BEETLE_SOC/GCC_ARM/mbed-os-prettyoutput.bin And the old behaviour of displaying the memap table can be brought back by passing the --stats-depth parameter

cmonr · 2018-01-26T21:12:00Z

/morph build

mbed-ci · 2018-01-26T21:59:23Z

Build : SUCCESS

Build number : 979
Build artifacts/logs : http://mbed-os.s3-website-eu-west-1.amazonaws.com/?prefix=builds/5929/

Triggering tests

/morph test
/morph uvisor-test
/morph export-build

cmonr · 2018-01-26T22:26:13Z

uVisor tests are experiencing an anomaly. @0xc0170 has ping'd the team to determine how to resolve.

mbed-ci · 2018-01-27T00:35:27Z

Exporter Build : FAILURE

Build number : 671
Build artifacts/logs : http://mbed-os.s3-website-eu-west-1.amazonaws.com/?prefix=builds/exporter/5929/

mbed-ci · 2018-01-27T02:51:02Z

Test : SUCCESS

Build number : 804
Test logs :http://mbed-os-logs.s3-website-us-west-1.amazonaws.com/?prefix=logs/5929/804

0xc0170 · 2018-01-29T08:42:17Z

/morph uvisor-test
/morph export-build

mbed-ci · 2018-01-29T11:41:42Z

Exporter Build : SUCCESS

Build number : 679
Build artifacts/logs : http://mbed-os.s3-website-eu-west-1.amazonaws.com/?prefix=builds/exporter/5929/

kegilbert reviewed Jan 25, 2018

View reviewed changes

geky force-pushed the prettybars branch from a4bfd01 to a69a665 Compare January 25, 2018 01:06

geky force-pushed the prettybars branch from a69a665 to 834e3d0 Compare January 25, 2018 01:16

0xc0170 added the needs: review label Jan 25, 2018

geky force-pushed the prettybars branch from 834e3d0 to dc4d2bc Compare January 25, 2018 17:05

geky force-pushed the prettybars branch from dc4d2bc to b2a5ce1 Compare January 25, 2018 17:24

theotherjimmy suggested changes Jan 25, 2018

View reviewed changes

geky force-pushed the prettybars branch from b2a5ce1 to 471d99c Compare January 25, 2018 17:46

theotherjimmy approved these changes Jan 25, 2018

View reviewed changes

0xc0170 approved these changes Jan 26, 2018

View reviewed changes

cmonr added needs: CI and removed needs: review labels Jan 26, 2018

0xc0170 added ready for merge and removed needs: CI labels Jan 29, 2018

cmonr added the release-version: 5.7.5 label Jan 29, 2018

cmonr merged commit 702eaff into master Jan 29, 2018

cmonr removed the ready for merge label Jan 29, 2018

MikeDK mentioned this pull request Jan 30, 2018

How to enable pretty bars on targets where they don't actually work? #5968

Closed

geky deleted the prettybars branch February 1, 2018 21:12

This was referenced Feb 21, 2018

tools: Bring back memap output by default #6155

Closed

Revert "Added pretty bar printing for compile output" #6197

Merged

jeromecoutant mentioned this pull request Mar 12, 2018

Pretty bars feature optimisation with dual RAM targets #6334

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add pretty bars for compile output #5929

Add pretty bars for compile output #5929

geky commented Jan 25, 2018 •

edited

Loading

kegilbert Jan 25, 2018

geky Jan 25, 2018 •

edited

Loading

kegilbert Jan 25, 2018

kegilbert commented Jan 25, 2018

geky commented Jan 25, 2018

drewcassidy commented Jan 25, 2018

0xc0170 commented Jan 25, 2018

geky commented Jan 25, 2018

drewcassidy commented Jan 25, 2018

geky commented Jan 25, 2018

geky commented Jan 25, 2018 •

edited

Loading

theotherjimmy commented Jan 25, 2018

geky commented Jan 25, 2018

theotherjimmy Jan 25, 2018

geky Jan 25, 2018

theotherjimmy Jan 25, 2018

theotherjimmy Jan 25, 2018

geky Jan 25, 2018 •

edited

Loading

theotherjimmy Jan 25, 2018

theotherjimmy Jan 25, 2018

theotherjimmy Jan 25, 2018

theotherjimmy Jan 25, 2018

geky Jan 25, 2018

theotherjimmy Jan 25, 2018

theotherjimmy commented Jan 25, 2018

cmonr commented Jan 26, 2018

mbed-ci commented Jan 26, 2018

cmonr commented Jan 26, 2018

mbed-ci commented Jan 27, 2018

mbed-ci commented Jan 27, 2018

0xc0170 commented Jan 29, 2018

mbed-ci commented Jan 29, 2018

	# TODO add tty detection, and width detection probably
	WIDTH = 72
	try:
	# NOTE this only works on linux
	import sys, fcntl, termios, struct
	height, width, _, _ = struct.unpack('HHHH',
	fcntl.ioctl(sys.stdout.fileno(), termios.TIOCGWINSZ,
	struct.pack('HHHH', 0, 0, 0, 0)))
	WIDTH = min(width, WIDTH)
	except:
	pass

Add pretty bars for compile output #5929

Add pretty bars for compile output #5929

Conversation

geky commented Jan 25, 2018 • edited Loading

Choose a reason for hiding this comment

geky Jan 25, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kegilbert commented Jan 25, 2018

geky commented Jan 25, 2018

drewcassidy commented Jan 25, 2018

0xc0170 commented Jan 25, 2018

geky commented Jan 25, 2018

drewcassidy commented Jan 25, 2018

geky commented Jan 25, 2018

geky commented Jan 25, 2018 • edited Loading

theotherjimmy commented Jan 25, 2018

geky commented Jan 25, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

geky Jan 25, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

theotherjimmy commented Jan 25, 2018

cmonr commented Jan 26, 2018

mbed-ci commented Jan 26, 2018

Triggering tests

cmonr commented Jan 26, 2018

mbed-ci commented Jan 27, 2018

mbed-ci commented Jan 27, 2018

0xc0170 commented Jan 29, 2018

mbed-ci commented Jan 29, 2018

geky commented Jan 25, 2018 •

edited

Loading

geky Jan 25, 2018 •

edited

Loading

geky commented Jan 25, 2018 •

edited

Loading

geky Jan 25, 2018 •

edited

Loading