Revisit slugs

[candlerb]

I'd like to bring up the issue of slugs again, as a follow-up to #8113 (closed). I'm raising this as a discussion for now, but will reformat as formal feature request(s) if there's any chance of it being accepted.

The main inconvenience with slugs is that they are mandatory: e.g. when creating a site via the API, if you don't provide one, you get django.core.exceptions.ValidationError: {'slug': ['This field cannot be blank.']}. Hence you need to duplicate the slug-synthesis logic (which currently sits in Javascript) in Python.

And yet, whilst being so "important" that they are mandatory, they are not even shown in the UI, e.g. /dcim/sites/123/ doesn't show the slug (although the "Edit" page does) (They are shown in light grey, at top right corner; just not in the main attributes table)

Part 1

My outline proposal is:

1. Make slugs optional
   • The slug column in each table is changed to nullable
   • UI changed to make them optional, and doesn't auto-fill them [^1]
2. Keep the unique constraint (that's what's different to Make slug fields optional for all models #8113)

At this point, if you choose to use slugs, they are still valid as alternate unique keys (for those models that have them anyway).

I notice there are a handful of places where slugs are used in HTML templates:

$ grep -iR slug netbox/templates
netbox/templates/circuits/provider.html:                            <a href="{% url 'circuits:circuit_list' %}?provider={{ object.slug }}">{{ circuits_table.rows|length }}</a>
netbox/templates/dcim/devicetype.html:                                <small class="text-muted">{{ object.slug }}</small>
netbox/templates/dcim/device_edit.html:              <a href="{% url 'dcim:devicebay_depopulate' pk=object.parent_bay.pk %}" title="Regenerate Slug" class="btn btn-danger d-inline-flex align-items-center">
netbox/templates/generic/object.html:            {% if object.slug %}({{ object.slug }}){% endif %}
netbox/templates/ipam/aggregate.html:                <a href="{% url 'ipam:aggregate_list' %}?rir={{ object.rir.slug }}">{{ object.rir }}</a>
netbox/templates/ipam/vlan.html:                                    <a href="{% url 'ipam:vlan_list' %}?role={{ object.role.slug }}">{{ object.role }}</a>
netbox/templates/ipam/asn.html:                <a href="{% url 'ipam:asn_list' %}?rir={{ object.rir.slug }}">{{ object.rir }}</a>
netbox/templates/extras/tag.html:                      <a href="{% url viewname %}?tag={{ object.slug }}">{{ object_type.item_count }}</a>

These would have to be changed to use ids - e.g. provider_id={{ object.id }} instead of provider={{ object.slug }}, to ensure they work for providers without slugs.

Alternatively, slugs could remain mandatory for a few models. I note that tags are especially awkward to use without slugs. (I don't think anyone wants to write a REST API call with ?tag_id=123 when they could use ?tag=snmp)

Part 2

As a separate feature request, I'd like to see that the CSV/YAML upload accepts slugs as well as Netbox sequential IDs as a way to do upserts. That is: if you provide a slug (but not an ID) and there's an existing record with that slug, it gets updated instead of creating a new one. Since slugs already exist for some models and are already unique, this is an independent FR.

Part 3

Add slugs, in the same optional form, to all other models - e.g. Device. This makes them useful for syncing records between systems - you can choose to put in a UUID for instance - but since they're optional, nobody who doesn't want them is inconvenienced in any way. For completeness, allow filters like device=<slug> as well as device_id=<id>

Whether or not you accept part 2 or part 3, I think part 1 stands on its own.

---

[^1] Ideally, there would be some pluggable pre-save logic to auto-create slugs. Then you could plug in the existing logic to derive a slug from the name, or generate a UUID, or whatever you want. I'm not sure if custom validators are allowed the modify the model they are validating.

In the Edit page, the slug should not be changed unless explicitly edited - but that's already the case today.

Having a button next to the slug field, to auto-fill it using the existing Javascript logic, might also help people who like the current arrangement.

Alternative

As an alternative to address the API inconvenience: if slugs stay as they are today, then I'd suggest adding the auto-slug-creation logic to the models to be used when the slug is empty. In this way, if you try to create a site but don't provide a slug, it will supply one for you. In the case of a conflict, it would have to use <slug>-<suffix>.

[candlerb]

Workaround:

def slugify(model, name, chars=50):
    base = name
    base = re.sub(r'[^-.\w\s]', '', base)       # Remove unneeded chars
    base = re.sub(r'^[\s.]+|[\s.]+$', '', base) # Trim leading/trailing spaces
    base = re.sub(r'[-.\s]+', '-', base)        # Convert spaces and decimals to hyphens
    base = base.lower()                         # Convert to lowercase
    slug = base[0:chars]                        # Trim to first chars
    for i in range(5):
        if model.objects.filter(slug=slug).count() > 0:
            slug = "%s-%06x" % (base[0:chars-7], random.randrange(0,0x1000000))
        else:
            return slug
    else:
        raise AbortScript("It's not your lucky day - unable to create a unique slug")

...

    def run(self, data, commit):
        site = Site(
            name=data['site_name'],
            slug=slugify(Site, data['site_name']),
            ...

(Note: Site already has a globally unique name, so the slug logic will rarely find conflicts. But it will if you had one site called "foobar" and another called "FooBar" or "foo@bar", for example)

[abhi1693]

I didn't get time to read your proposal thoroughly but just to point out, slugs are displayed on the object views in the top right corner of the model supports it

[jcollie]

I personally would eliminate slugs altogether. Objects already have (per-device model) unique identifier. I personally never use slugs in the scripts that I write, except when I have to generate them to satisfy the API (I may switch to just generating a UUID instead of duplicating the name-to-slug javascript code).

[abhi1693]

While I understand your concerns about the redundancy of slugs when unique identifiers are already in place, there are still several reasons why slugs can be useful and should be retained. Here are a few use cases to illustrate their benefits:

1. Slugs provide a more readable and easily understandable representation of the object, especially when compared to UUIDs or other unique identifiers. We use particularly use these to find device types as these always generate the same slugs no matter how the name is written due to the strict rules it follows.

2. Filterability and searchability: Slugs can be more easily indexed and searched for within the system, as they typically represent a meaningful aspect of the object. For example, a slug derived from the object's name or description makes it easier for users to search and filter relevant items. UUIDs, on the other hand, are not easily searchable or filterable due to their random nature.

3. Slugs can be useful when integrating with external systems, APIs, or tools that expect human-readable and URL-friendly identifiers. In such cases, slugs can serve as a common ground for communication, ensuring seamless integration. In this case, it helps us to bind different platforms together as we dont need to either hunt for IDs or filter using complex scenarios.

4. Slugs can be helpful in data visualization, as they provide meaningful labels for objects being represented in charts, graphs, or tables. This can enhance the understandability of the data being presented and make it easier for users to interpret. This is where our Grafana charts shine for us.

In conclusion, despite the presence of unique identifiers such as UUIDs, slugs still offer numerous advantages in terms of readability, searchability, consistency, and integration. These benefits make them a valuable component in a variety of use cases, and thus, I would argue that they should be retained.

[jcollie]

Slugs, since they are mutable, are absolutely awful for integration with other systems that require a fixed unique ID. Searching based on the slug makes no sense at all to me. Search terms should be either in the name, description or some other field in the object. Data visualization should be using the name attribute. If your name attributes aren't useful for data visualization that doesn't seem like a NetBox problem or a problem that slugs should be solving. And slugs won't always be generated the same way if you already have a slug in the system that conflicts with the way that slugs are generated.

[abhi1693]

Slugs, since they are mutable, are absolutely awful for integration with other systems that require a fixed unique ID.

I have systems that do rely on slugs, especially that's taking care of the automation peices for us. Now, regarding the mutation all fields in netbox mutate with the exception of PK. But how many times do you refer something with hard-coded PK values (awful idea when integration is considered). One can say I can filter using name, description and other fields, but why when slugs do all that quite easily. Example: my automation looks for device types using slug rather than names because a name can be written with any case, can have multiple spaces, dashes, underscores, special characters etc. But if slugs (used correctly) will adhere to strict rules and will always generate deterministic values.

1. name= AP 7000, slug= ap7000
2. name= AP 7000, slug= ap7000

When you look at the name as a human, you probably wont see differences but a program will treat them as different values. I'm not saying both values should exist simultaneously, but you can have multiple spaces before, after and in-between characters. How do you filter based on such values? There is no deterministic way unless you consider slugs here which creates the same string no matter what and now can be used to filter the object. The models in NetBox provides slug fields in all the right models (no more and no less). A slug on a device does not make sense but does on the device type which is what is really provided here and in other places.

Data visualization should be using the name attribute. If your name attributes aren't useful for data visualization that doesn't seem like a NetBox problem or a problem that slugs should be solving.

Names are terrible when you want to annotate your charts because they aren't consistent. For seeing things on legend, I may prefer a name but to annotate my chart, I'd consider a slug or a tag (slug) in this case. When such systems integrate with external sources, you shouldn't use names but those tags (slugs) on the charts to fetch relevant data. Here again, the slugs provide deterministic filtering capability.

You cannot simply say slugs are not netbox problem, that statement can be applied to several other nuances in NetBox that probably applies to my use case but not yours. Just because something is provided for convience and ease of use should not also be considered useless in my opinion.

Now, when you consider description, humans can write whatever they want in those fields and when you work on automation alot, it becomes that much diffcult to search for things. Using custom validators to apply in the 100s of different places would just make things more difficult to maintain and will cause furstation among users. There may even be exceptions and the validator may not allow such anomalies.

And slugs won't always be generated the same way if you already have a slug in the system that conflicts with the way that slugs are generated.

Slugs aren't generated in NetBox. You can modify it to anything you want. It's not enforcing anything specific from the ruleset on the humans. If you point out the UI for us, the slug field is just providing a one-off value that follows those rules and is left on the humans to put whatever they want.

Note: I'm not saying slugs are perfect in netbox but they still provide good functionality. There maybe ways to improve it further but total removal atleast for my organization and several other projects is not desirable.

[mtinberg]

I use slugs, to make the API call or URL more readable, when composing DNS compatible names, or when building a hash/dictionary of items to normalize existence checks when comparing with other IT systems, eg. the "Basketweaving Department" building is going to be "basketweaving" in DNS and other IT systems, which is easily represented by the slug, so auditing "switch01-basketweaving-access" device makes sense or referencing /dcim/devices/?site=basketweaving instead of ?site_id=3. Kind of like how export CSV and import CVS are different because human-readable and machine-readable representations are a little different. — Mark Tinberg ***@***.***> Division of Information Technology-Network Services University of Wisconsin-Madison

…

________________________________ From: Jeffrey C. Ollie ***@***.***> Sent: Friday, April 21, 2023 10:09 AM To: netbox-community/netbox ***@***.***> Cc: Subscribed ***@***.***> Subject: Re: [netbox-community/netbox] Revisit slugs (Discussion #12315) I personally would eliminate slugs altogether. Objects already have (per-device model) unique identifier. I personally never use slugs in the scripts that I write, except when I have to generate them to satisfy the API (I may switch to just generating a UUID instead of duplicating the name-to-slug javascript code). — Reply to this email directly, view it on GitHub<#12315 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AAS7UM7ANZHWLF5LYIVTIATXCKPMHANCNFSM6AAAAAAXGYYWPQ>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[markkuleinio]

I don't "use" slugs. They can be modified on the fly so I don't see them useful for linking between systems or documents. The NetBox GUI-autogenerated slugs are also not visually compatible with non-English strings ("Ääliö, älä lyö, ööliä läikkyy" -> "li-l-ly-li-likkyy").

But, when they are mandatory, all my apps just generate them as a random 8-character strings.

[candlerb]

BTW, I came across a case where slugs aren't globally unique: device types. It's only the (manufacturer,slug) combination which is unique. That makes the slug not directly usable as a slug.

It would be more consistent IMO if device type slugs were forced to be unique, even if they had to be e.g. "cisco-ncc-1701" and "ufp-ncc-1701" for two manufacturers with a model "ncc-1701".

[llamafilm]

Are slugs really mandatory? I can create a site without a slug using a script:

>>> s = Site(name='la-lai')
>>> s.save()
>>> s.slug
''

It looks like it made the slug an empty string. Now if I try to edit this site in the GUI, it won't let me save:

I'll refrain from opining on whether to remove slugs completely, but if they are going to be mandatory in the GUI, then for consistency they should also be mandatory in a script. Trying to create an object without one should either throw an exception or (preferably) automatically slugify the name as the GUI does.

[kkthxbye-code]

You have to call full_clean() on the object before saving, otherwise no validation is executed.

[llamafilm]

Thanks, you are correct in this case. It's odd that Site and IPAddress behave differently in this respect.

[candlerb]

IPAddresses don't have slugs.