Implement best practices for linkerd-viz app

[LolloneS]

Coming from here.

The goal of this issue is to have the App eventually adhere to standard cloud-native best-practices. The points below are to be considered non-exhaustive. Feel free to add some that make sense and remove some that don't. If this issue does not make sense (e.g. is a duplicate or covers a deprecated app) please close it.

Expected outcome:

• the app has properly set PDBs (as percentages)
• the app can scale horizontally based on Custom Metrics (if it makes sense)
• the app runs with more than 1 replica by default (if it needs HA)
• the app does not use emptyDir volumes if not strictly necessary

[mcharriere]

We'll contribute PDB to upstream: linkerd/linkerd2#11248

[mcharriere]

the app can scale horizontally based on Custom Metrics (if it makes sense)

It doesn't make much sense in this case, as scaling is largely dependent on cluster size / tap usage. We might need to revisit scaling in the future, depending on customer use.

the app does not use emptyDir volumes if not strictly necessary

it does require it indirectly, as all linkerd.

[mcharriere]

1.2.0 released including PDB. Upstream PR is still open, hopefully it gets merged.