Exploration: supported languages as "singleton values": macros #53

helixbass · 2023-07-11T18:32:54Z

In this PR:

add macros for "safely" generating fixed-size "all supported languages" collections
a couple ergonomic improvements related to these per comments on Exploration: supported languages as "singleton values" #52

To test:
You should see the fixed_map!() macro fail if you try to omit any keys, pass any spurious key/values or not pass corresponding values of the expected type
You should see the by_supported_language!() macro fail if you omit any supported languages or pass any unknown keys

Based on supported-languages-fixed-size

helixbass · 2023-07-11T18:33:45Z

proc_macros/Cargo.toml

+proc-macro = true
+
+[dependencies]
+Inflector = "0.11.4"


A capitalized crate name?

Almost seems ironic since what this crate is doing is case conversion

Ha, maybe it's a weird joke.

helixbass · 2023-07-11T18:36:03Z

proc_macros/src/lib.rs

+    let len = variants.len();
+    quote! {
+        #[derive(Default)]
+        pub struct #collection_type_name<T>([T; #len]);


Per #52 (comment), add a newtype wrapper around the fixed-size collection type (eg BySupportedLanguage) for the purpose of exposing "HashMap-ish" iteration methods

I actually made the changes corresponding to those comments "statically" before starting the macro stuff so if you'd rather see a pre-macroization version of them you could look at the diff as of f672cca

helixbass · 2023-07-11T18:38:32Z

proc_macros/src/lib.rs

+    }
+}
+
+fn get_iter_implementations(


Per above comment this is implementing "HashMap-style" .iter()/.values() on eg BySupportedLanguage

helixbass · 2023-07-11T18:39:33Z

proc_macros/src/lib.rs

+) -> proc_macro2::TokenStream {
+    let len = variants.len();
+    quote! {
+        impl<T> Deref for #collection_type_name<T> {


Added a Deref implementation along with the newtype wrapper (I think it's currently only getting used to call the underlying .len())

helixbass · 2023-07-11T18:43:19Z

proc_macros/src/lib.rs

+) -> proc_macro2::TokenStream {
+    let macro_name = format_ident!("{}", collection_type_name.to_string().to_snake_case());
+    quote! {
+        #[macro_export]


Ok so this is sort of funny, the way that I was able to get a "generated" eg by_supported_language!() macro (that needed to be basically implemented as a proc macro, but per #52 (comment) I don't think you could easily just "generate and use a new proc macro"?) was to generate it as a declarative macro which basically delegates to an existing "static" by_fixed_map!() proc macro but has the things specific to "this fixed map" (ie the variant names and collection type name eg BySupportedLanguage) "pre-curried" as arguments to it (by hardcoding them into the declarative macro body basically)

It's sort of funny that I believe those pre-curried arguments just get passed along from one to the other "as token trees", so they can eg reference non-existing identifiers

Ha yes, that's kind of ridiculous, but cool that it works.

helixbass · 2023-07-11T18:44:34Z

proc_macros/src/lib.rs

+            value_mapping_content.parse::<Token![=>]>()?;
+            let value: Expr = value_mapping_content.parse()?;
+            value_mapping.insert(key, value);
+            if value_mapping_content.is_empty() {


On one of these macros I think I required a trailing comma but there are certainly ways to avoid that eg I copied this pattern from the syn::ExprArray Parse implementation

helixbass · 2023-07-11T18:45:14Z

proc_macros/src/lib.rs

+    } = parse_macro_input!(input as ByFixedMapArgs);
+
+    if value_mapping.len() != variants.len() {
+        panic!("Incorrect variants");


These could be more helpful and tell you eg which variant(s) were missing or what invalid variant names it encountered

helixbass · 2023-07-11T18:46:19Z

src/language.rs

-    ]
-});
+impl SupportedLanguage {
+    pub fn language(&self) -> Language {


Per #52 (comment), move these onto methods on SupportedLanguage

helixbass · 2023-07-11T18:54:19Z

src/language.rs

-    "lua",
-];
+static SUPPORTED_LANGUAGE_LANGUAGES: Lazy<BySupportedLanguage<Language>> = Lazy::new(|| {
+    by_supported_language!(


I think I sort of like this approach of exposing a secondary macro for allowing you to create new "fixed-size collections" (as opposed to eg specifying all of the tree_sitter::Language's and "names for ignore select" in the body of the main fixed_map!() invocation and then either generating the same thing as this under the hood or possible generating "fat SupportedLanguage structs" (like how it used to have language/name_for_ignore_select as struct fields rather than these separate top-level data structures))?

I guess the downsides could be (a) in terms of code readability if one thought it were clearer to read "one big declaration" of all of the info/"fields" for all of the supported languages and (b) in terms of runtime if for some reason it were more performant ("cache locality"...?) (and if this were then actually a practical performance issue, I'm really just theorizing) to actually have those fields co-located as part of their relevant "SupportedLanguage struct" (vs being co-located with other instances of "that same field" like this)?

helixbass · 2023-07-11T18:57:50Z

src/project_file_walker.rs

    } else {
-        for name_for_ignore_select in SUPPORTED_LANGUAGE_NAMES_FOR_IGNORE_SELECT.iter() {


The top-level SUPPORTED_LANGUAGE_LANGUAGES/SUPPORTED_LANGUAGE_NAMES_FOR_IGNORE_SELECT can really be considered "private"/"implementation details" (and I did in fact make them not pub) since you can achieve eg here iterating over them by iterating over the main ALL_SUPPORTED_LANGUAGES and then accessing the corresponding relevant "field" via the helper methods (eg here .name_for_ignore_select())

…s-fixed-size-macro

peterstuart · 2023-07-13T20:21:33Z

proc_macros/Cargo.toml

+proc-macro = true
+
+[dependencies]
+Inflector = "0.11.4"


Ha, maybe it's a weird joke.

peterstuart · 2023-07-13T20:22:47Z

proc_macros/src/lib.rs

+                "name" => {
+                    name = Some(input.parse::<Ident>()?);
+                }
+                "variants" => {


Whoa, interesting. I guess I haven't seen this kind of stuff in a proc-macro before.

Ya I was sort of "winging it" off of picturing the macro! { key => value, other_key => other_value } "invocation style"

peterstuart · 2023-07-13T20:23:25Z

proc_macros/src/lib.rs

+    let collection_type_name = format_ident!("By{name}");
+    let all_variants_collection_name = format_ident!(
+        "ALL_{}",
+        name.to_string().to_plural().to_screaming_snake_case()


I love the name "screaming snake case", and it seems like most people haven't even heard of it 😄

peterstuart · 2023-07-13T20:26:42Z

proc_macros/src/lib.rs

+            next_index: usize,
+        }
+
+        impl<'collection, T> #iter_struct_name<'collection, T> {


I never really thing about how Iterator is implemented on structs that exist just for that purpose.

Right I haven't read that many implementations of them (eg of whatever the types are returned by Vec::into_iter() etc) but I think they just tend to be little "state machines"?

peterstuart · 2023-07-13T20:28:48Z

proc_macros/src/lib.rs

+) -> proc_macro2::TokenStream {
+    let macro_name = format_ident!("{}", collection_type_name.to_string().to_snake_case());
+    quote! {
+        #[macro_export]


Ha yes, that's kind of ridiculous, but cool that it works.

peterstuart · 2023-07-13T20:30:13Z

proc_macros/src/lib.rs

+            let key: Ident = value_mapping_content.parse()?;
+            value_mapping_content.parse::<Token![=>]>()?;
+            let value: Expr = value_mapping_content.parse()?;
+            value_mapping.insert(key, value);


Worth checking that it's not already present? (Unless that's handled elsewhere.)

Ah ya that makes sense, updated

peterstuart · 2023-07-13T20:32:06Z

src/language.rs

+        TreeSitterQuery => tree_sitter_query::language(),
+        Json => tree_sitter_json::language(),
+        Css => tree_sitter_css::language(),
+        Lua => tree_sitter_lua::language(),


How would you feature-gate these (if you wanted to eventually), since you can't use the normal annotation in a macro. (Although I suppose you could make the macro understand that annotation?)

Although I suppose you could make the macro understand that annotation?

Ya per I think a comment on a subsequent PR (where I did cpp -> c++), if I parse into a baked-in syn type eg syn::ExprArray it parses annotations for me

But... this isn't real Rust syntax so couldn't use that directly

Maybe that would be an excuse to mimic how eg syn implements .parse::<ExprArray>() but have the "item" parsing be different or whatever

helixbass added 5 commits July 11, 2023 11:47

make methods

c4d3488

newtype wrapper

f672cca

proc macro for most of it

2d41e8f

macro

9016f8d

hide

67a94f9

helixbass commented Jul 11, 2023

View reviewed changes

Merge branch 'supported-languages-fixed-size' into supported-language…

669e77c

…s-fixed-size-macro

helixbass mentioned this pull request Jul 12, 2023

More languages #50

Merged

Merge branch 'supported-languages-fixed-size' into supported-language…

cfa3b3d

…s-fixed-size-macro

peterstuart approved these changes Jul 13, 2023

View reviewed changes

Base automatically changed from supported-languages-fixed-size to master July 13, 2023 21:23

guard

243a2a0

helixbass merged commit 7873e30 into master Jul 13, 2023
2 checks passed

helixbass deleted the supported-languages-fixed-size-macro branch July 13, 2023 21:42

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Exploration: supported languages as "singleton values": macros #53

Exploration: supported languages as "singleton values": macros #53

helixbass commented Jul 11, 2023

helixbass Jul 11, 2023

peterstuart Jul 13, 2023

helixbass Jul 11, 2023

helixbass Jul 11, 2023

helixbass Jul 11, 2023

helixbass Jul 11, 2023

peterstuart Jul 13, 2023

helixbass Jul 11, 2023

helixbass Jul 11, 2023

helixbass Jul 11, 2023

helixbass Jul 11, 2023

helixbass Jul 11, 2023

peterstuart Jul 13, 2023

peterstuart Jul 13, 2023

helixbass Jul 13, 2023

peterstuart Jul 13, 2023

peterstuart Jul 13, 2023

helixbass Jul 13, 2023

peterstuart Jul 13, 2023

peterstuart Jul 13, 2023

helixbass Jul 13, 2023

peterstuart Jul 13, 2023

helixbass Jul 13, 2023

		} else {
		for name_for_ignore_select in SUPPORTED_LANGUAGE_NAMES_FOR_IGNORE_SELECT.iter() {

Exploration: supported languages as "singleton values": macros #53

Exploration: supported languages as "singleton values": macros #53

Conversation

helixbass commented Jul 11, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment