New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: different way to specify exported symbols #462

Closed
andrewrk opened this Issue Sep 10, 2017 · 3 comments

Comments

Projects
None yet
4 participants
@andrewrk
Member

andrewrk commented Sep 10, 2017

Problem: We have some messy complications regarding the difference between pub and export.

  • pub, export, and unqualified are the 3 visibility modifiers. Pub makes sense. But if it's export then it goes into the global namespace as well as being exposed in the current module.
  • export automatically changes the calling convention of a function to C calling convention, which is represented in type syntax as extern fn()
  • extern fn(); also changes calling convention to CCC.
  • We also have nakedcc, stdcallcc, coldcc. But no ccc.
  • exported functions have special requirements such as disallowing comptime parameters and use of certain types.
  • when you @import a module which exported a function, that function name is meant for the global namespace, but you're going to call it through a module namespace.
  • we have extern fn() { ... } for a function definition that wants the C calling convention but does not need to be exported.
  • we also want to export types like structs and enums. That looks like export const Foo = extern struct { ... }.
  • aliases is a thing. #420

Here's the proposal:

  • Add this syntax:
export name = value;
  • Remove export from functions and global variables. (and generally remove it as a visibility modifier)
  • Add ccc as a calling convention. Use ccc fn() instead of extern fn() for the type syntax.
  • When you export a function, it gives compile errors if the function disobeys the exported function rules. Same with types.

I think this solves all the problems.

@andrewrk andrewrk added this to the 0.2.0 milestone Sep 10, 2017

@kyle-github

This comment has been minimized.

Show comment
Hide comment
@kyle-github

kyle-github Sep 10, 2017

When I made the comments last night I had not put together all the parts. It seems like there are a few mostly-orthogonal things here:

  • visibility from users of the module/namespace.
  • visibility at a global level (perhaps a degenerate condition of the above?)
  • calling convention

The reason I say mostly-orthogonal is, as you point out, that you need to have both a "global" visibility on a function and have it use the C calling convention in order for C code to call it. So the new export syntax above is only for things visible to C?

As I understand it (and maybe I am missing something important due to my lack of experience with Zig!), a C-callable function differs in two ways from a standard Zig function. It must be visible at the global level and it must use the C calling convention. So you would still use "pub" for module-level entities, right?

Would there be a use case for global entities that are not C-callable? If so, maybe then "export" becomes "global" and you retain the ccc part??

(I moved the original incoherent ideas over to the forum where they belong.)

kyle-github commented Sep 10, 2017

When I made the comments last night I had not put together all the parts. It seems like there are a few mostly-orthogonal things here:

  • visibility from users of the module/namespace.
  • visibility at a global level (perhaps a degenerate condition of the above?)
  • calling convention

The reason I say mostly-orthogonal is, as you point out, that you need to have both a "global" visibility on a function and have it use the C calling convention in order for C code to call it. So the new export syntax above is only for things visible to C?

As I understand it (and maybe I am missing something important due to my lack of experience with Zig!), a C-callable function differs in two ways from a standard Zig function. It must be visible at the global level and it must use the C calling convention. So you would still use "pub" for module-level entities, right?

Would there be a use case for global entities that are not C-callable? If so, maybe then "export" becomes "global" and you retain the ccc part??

(I moved the original incoherent ideas over to the forum where they belong.)

@PavelVozenilek

This comment has been minimized.

Show comment
Hide comment
@PavelVozenilek

PavelVozenilek Sep 10, 2017

I have related proposal. Modules (or packages) provide several features:

  1. Fast compilation, usually by not standing in the way.
  2. Namespacing. Most common use but IMHO overrated. C++ does it wrong, Nim better.
  3. Encapsulation. C/C++ has only very limited form - statics and anonymous namespaces.
  4. In the most extreme interpretation modules also deal with dynamically loaded code.

Here's the proposal about encapsulation:

  1. Source file is module. foo.zig is module foo (together with its directory path). It could export functions, these are visible according to the rules.
  2. If subdirectory named foo exists in the same place as foo.zig then this subdirectory and everything inside it is private implementation of the foo module.
  3. Private implementation of a module has no chance to escape out, regardless of their export/pub annotations. It could be invoked only from the foo module but by nobody else.

This way one could:

  1. Download 3rd party library, put it in private, wrap and export the desired parts. The un-needed rest of the library code won't be visible. There will be no name clashes. Library code would not need to be touched at all.
  2. Clone existing code, make modifications and place the new version as private implementation somewhere. Again - no name clashes possible. This could be used e.g to compare performance between different versions.
  3. Just looking on file names would be enough to know what is the interface (file foo.zig) and what is private implementation (foo directory). Time needed to grok a system would be reduced.
  4. Experiment and explore safely. Source files could be freely copied or cloned, without the need to make changes inside, and without fears of silently breaking something.

PavelVozenilek commented Sep 10, 2017

I have related proposal. Modules (or packages) provide several features:

  1. Fast compilation, usually by not standing in the way.
  2. Namespacing. Most common use but IMHO overrated. C++ does it wrong, Nim better.
  3. Encapsulation. C/C++ has only very limited form - statics and anonymous namespaces.
  4. In the most extreme interpretation modules also deal with dynamically loaded code.

Here's the proposal about encapsulation:

  1. Source file is module. foo.zig is module foo (together with its directory path). It could export functions, these are visible according to the rules.
  2. If subdirectory named foo exists in the same place as foo.zig then this subdirectory and everything inside it is private implementation of the foo module.
  3. Private implementation of a module has no chance to escape out, regardless of their export/pub annotations. It could be invoked only from the foo module but by nobody else.

This way one could:

  1. Download 3rd party library, put it in private, wrap and export the desired parts. The un-needed rest of the library code won't be visible. There will be no name clashes. Library code would not need to be touched at all.
  2. Clone existing code, make modifications and place the new version as private implementation somewhere. Again - no name clashes possible. This could be used e.g to compare performance between different versions.
  3. Just looking on file names would be enough to know what is the interface (file foo.zig) and what is private implementation (foo directory). Time needed to grok a system would be reduced.
  4. Experiment and explore safely. Source files could be freely copied or cloned, without the need to make changes inside, and without fears of silently breaking something.
@andrewrk

This comment has been minimized.

Show comment
Hide comment
@andrewrk

andrewrk Sep 13, 2017

Member

This proposal also makes things like this more clear:

export nakedcc fn __chkstk() {
    @setDebugSafety(this, false);

    if (comptime builtin.os == builtin.Os.windows) {
        if (comptime builtin.arch == builtin.Arch.x86_64) {
            asm volatile (
                \\         push   %%rcx
                \\         cmp    $0x1000,%%rax
                \\         lea    16(%%rsp),%%rcx     // rsp before calling this routine -> rcx
                \\         jb     1f
                \\ 2:
                \\         sub    $0x1000,%%rcx
                \\         test   %%rcx,(%%rcx)
                \\         sub    $0x1000,%%rax
                \\         cmp    $0x1000,%%rax
                \\         ja     2b
                \\ 1:
                \\         sub    %%rax,%%rcx
                \\         test   %%rcx,(%%rcx)
                \\ 
                \\         lea    8(%%rsp),%%rax     // load pointer to the return address into rax
                \\         mov    %%rcx,%%rsp        // install the new top of stack pointer into rsp
                \\         mov    -8(%%rax),%%rcx    // restore rcx
                \\         push   (%%rax)           // push return address onto the stack
                \\         sub    %%rsp,%%rax        // restore the original value in rax
                \\         ret
            );
            unreachable;
        }
    }

    @setGlobalLinkage(__chkstk, builtin.GlobalLinkage.Internal);
}

Now it's:

comptime {
    if (builtin.os == builtin.Os.windows and builtin.arch == builtin.Arch.x86_64) {
        export __chkstk = windows_check_stack;
    }
}

nakedcc fn windows_check_stack() {
    @setDebugSafety(this, false);

    asm volatile (
        \\         push   %%rcx
        \\         cmp    $0x1000,%%rax
        \\         lea    16(%%rsp),%%rcx     // rsp before calling this routine -> rcx
        \\         jb     1f
        \\ 2:
        \\         sub    $0x1000,%%rcx
        \\         test   %%rcx,(%%rcx)
        \\         sub    $0x1000,%%rax
        \\         cmp    $0x1000,%%rax
        \\         ja     2b
        \\ 1:
        \\         sub    %%rax,%%rcx
        \\         test   %%rcx,(%%rcx)
        \\ 
        \\         lea    8(%%rsp),%%rax     // load pointer to the return address into rax
        \\         mov    %%rcx,%%rsp        // install the new top of stack pointer into rsp
        \\         mov    -8(%%rax),%%rcx    // restore rcx
        \\         push   (%%rax)           // push return address onto the stack
        \\         sub    %%rsp,%%rax        // restore the original value in rax
        \\         ret
    );
    unreachable;
}

And now that symbol __chkstk is available on non windows for use. This also occurs with main in std/special/bootstrap.zig which can cause a confusing symbol collision error:

export fn main(c_argc: i32, c_argv: &&u8, c_envp: &?&u8) -> i32 {
    if (!want_main_symbol) {
        @setGlobalLinkage(main, builtin.GlobalLinkage.Internal);
        unreachable;
    }

    callMain(usize(c_argc), c_argv, c_envp) %% return 1;
    return 0;
}

becomes

comptime {
    if (want_main_symbol) {
        export main = c_main_entry_point;
    }
}
ccc fn c_main_entry_point(c_argc: i32, c_argv: &&u8, c_envp: &?&u8) -> i32 {
    callMain(usize(c_argc), c_argv, c_envp) %% return 1;
    return 0;
}

and now exporting the main symbol is allowed for the user.

Member

andrewrk commented Sep 13, 2017

This proposal also makes things like this more clear:

export nakedcc fn __chkstk() {
    @setDebugSafety(this, false);

    if (comptime builtin.os == builtin.Os.windows) {
        if (comptime builtin.arch == builtin.Arch.x86_64) {
            asm volatile (
                \\         push   %%rcx
                \\         cmp    $0x1000,%%rax
                \\         lea    16(%%rsp),%%rcx     // rsp before calling this routine -> rcx
                \\         jb     1f
                \\ 2:
                \\         sub    $0x1000,%%rcx
                \\         test   %%rcx,(%%rcx)
                \\         sub    $0x1000,%%rax
                \\         cmp    $0x1000,%%rax
                \\         ja     2b
                \\ 1:
                \\         sub    %%rax,%%rcx
                \\         test   %%rcx,(%%rcx)
                \\ 
                \\         lea    8(%%rsp),%%rax     // load pointer to the return address into rax
                \\         mov    %%rcx,%%rsp        // install the new top of stack pointer into rsp
                \\         mov    -8(%%rax),%%rcx    // restore rcx
                \\         push   (%%rax)           // push return address onto the stack
                \\         sub    %%rsp,%%rax        // restore the original value in rax
                \\         ret
            );
            unreachable;
        }
    }

    @setGlobalLinkage(__chkstk, builtin.GlobalLinkage.Internal);
}

Now it's:

comptime {
    if (builtin.os == builtin.Os.windows and builtin.arch == builtin.Arch.x86_64) {
        export __chkstk = windows_check_stack;
    }
}

nakedcc fn windows_check_stack() {
    @setDebugSafety(this, false);

    asm volatile (
        \\         push   %%rcx
        \\         cmp    $0x1000,%%rax
        \\         lea    16(%%rsp),%%rcx     // rsp before calling this routine -> rcx
        \\         jb     1f
        \\ 2:
        \\         sub    $0x1000,%%rcx
        \\         test   %%rcx,(%%rcx)
        \\         sub    $0x1000,%%rax
        \\         cmp    $0x1000,%%rax
        \\         ja     2b
        \\ 1:
        \\         sub    %%rax,%%rcx
        \\         test   %%rcx,(%%rcx)
        \\ 
        \\         lea    8(%%rsp),%%rax     // load pointer to the return address into rax
        \\         mov    %%rcx,%%rsp        // install the new top of stack pointer into rsp
        \\         mov    -8(%%rax),%%rcx    // restore rcx
        \\         push   (%%rax)           // push return address onto the stack
        \\         sub    %%rsp,%%rax        // restore the original value in rax
        \\         ret
    );
    unreachable;
}

And now that symbol __chkstk is available on non windows for use. This also occurs with main in std/special/bootstrap.zig which can cause a confusing symbol collision error:

export fn main(c_argc: i32, c_argv: &&u8, c_envp: &?&u8) -> i32 {
    if (!want_main_symbol) {
        @setGlobalLinkage(main, builtin.GlobalLinkage.Internal);
        unreachable;
    }

    callMain(usize(c_argc), c_argv, c_envp) %% return 1;
    return 0;
}

becomes

comptime {
    if (want_main_symbol) {
        export main = c_main_entry_point;
    }
}
ccc fn c_main_entry_point(c_argc: i32, c_argv: &&u8, c_envp: &?&u8) -> i32 {
    callMain(usize(c_argc), c_argv, c_envp) %% return 1;
    return 0;
}

and now exporting the main symbol is allowed for the user.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment