Skip to content
This repository has been archived by the owner on Oct 12, 2022. It is now read-only.

Added std::array<> #2286

Merged
merged 12 commits into from
Nov 8, 2018
Merged

Added std::array<> #2286

merged 12 commits into from
Nov 8, 2018

Conversation

TurkeyMan
Copy link
Contributor

First and simplest of the bunch. I'd like to get this in to establish layout and styling for the others.

@dlang-bot
Copy link
Contributor

Thanks for your pull request and interest in making D better, @TurkeyMan! We are looking forward to reviewing it, and you should be hearing from a maintainer soon.
Please verify that your PR follows this checklist:

  • My PR is fully covered with tests (you can see the annotated coverage diff directly on GitHub with CodeCov's browser extension
  • My PR is as minimal as possible (smaller, focused PRs are easier to review than big ones)
  • I have provided a detailed rationale explaining my changes
  • New or modified functions have Ddoc comments (with Params: and Returns:)

Please see CONTRIBUTING.md for more information.


If you have addressed all reviews or aren't sure how to proceed, don't hesitate to ping us with a simple comment.

Bugzilla references

Your PR doesn't reference any Bugzilla issue.

If your PR contains non-trivial changes, please reference a Bugzilla issue or create a manual changelog.

Testing this PR locally

If you don't have a local development environment setup, you can use Digger to test this PR:

dub fetch digger
dub run digger -- build "master + druntime#2286"

@@ -46,6 +46,8 @@ SRCS=\
src\core\stdc\time.d \
src\core\stdc\wchar_.d \
\
src\core\stdcpp\array.d \
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why aren't any other the other existing files in here?


alias as_array this;

extern(D) size_type size() const nothrow @safe @nogc { return N; }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can just stick an extern(D): at the top. Also I would explain in the doc why those are extern(D), as it might be confusing to users/reviewers.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I thought about that... but then I thought it might be too easy to accidentally mis-attribute any proper extern's.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any proper extern(C++) will have it in front of the prototype, won't it ?
I remember doing this for a project of my own and not having to define any extern(C++) anyway because everything gets inlined.

Copy link
Contributor

@jacob-carlborg jacob-carlborg Aug 27, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, extern(D) is confusing to me.. Looking at this on my phone, didn’t scroll to the code right.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If these functions are extern(C++), they may conflict with actual C++ implementations of these functions. Locally defined functions should be D-mangled.

But what does it matter, you guys are rejecting it all anyway :/


extern(C++, std):

extern(C++, class) struct array(T, size_t N)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Documentation


module core.stdcpp.array;

alias array = std.array;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's not necessary

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It really is!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you elaborate ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TL;DR, Walter says.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TL;DR, slightly longer: you can't import 2 such modules because the namespace scopes conflict with eachother. Also, the namespace called 'std' conflicts with phobos. Basically, we just need to alias all these things out into the module scope (where they should be anyway) so that anything is accessible.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense.
Can you add a comment (not DDOC) to mention this ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure.

/**
* D header file for interaction with C++ std::array.
*
* Copyright: Manu Evans 2014 - 2018.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That'd be problematic I think

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True.. what's the proper copyright assignment?
I just copied what I found in neighbouring code.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To the "D Language Foundation"

@Geod24
Copy link
Member

Geod24 commented Aug 27, 2018

Can you add ABI tests ?

@TurkeyMan TurkeyMan force-pushed the std_array branch 2 times, most recently from caeb696 to 84c614b Compare August 27, 2018 04:09
@TurkeyMan
Copy link
Contributor Author

I don't really know how to test this. It involves interaction with all the possible C compilers, and the build/CI scripts are chaos.
Perhaps someone (you?) could amend this patch to include a framework for testing these STL patches, and then I can populate with tests and proliferate to vector/string/etc.

@JinShil
Copy link
Contributor

JinShil commented Aug 27, 2018

Why are we putting C++ bindings in druntime? Can't these exist in a Dub package that doesn't require the D Language Foundation to take ownership of it, and assume the burden of maintaining it?

@TurkeyMan
Copy link
Contributor Author

This was a D Language Foundation project from a while back. There was a hired employee working on this.
I think he quit, this was incomplete, and I'm resurrecting the effort.

I guess it feels natural that they exist beside core.stdc??

There's no way a dub package will maintain correct sync, and these files, possibly more than any other files in the distribution, definitely need to run their CI and tests together with the releases to guarantee ABI match.

@TurkeyMan
Copy link
Contributor Author

Anything less will be chaos. These bindings are of critical importance, and really need to 'just work'. An external repo will never keep sync. I don't think they can live anywhere else.

@JinShil
Copy link
Contributor

JinShil commented Aug 27, 2018

I realize someone decided long ago that they should be here in druntime; I'm challenging that decision.

I'm not convinced that a Dub repository would not work. Even if the D Language Foundation chose to take ownership of these bindings, they could exist in a different repository or https://github.com/D-Programming-Deimos/

I think the only reason why the C standard library bindings are here is because they are used by some of the druntime internals. If druntime has no need for these bindings internally, I don't think they belong here. Even if druntime did use them internally, there are other ways to go about that.

I ask that we strive to keep the scope of druntime to be just what is necessary to implement the D language, and nothing more.

@TurkeyMan
Copy link
Contributor Author

These files very closely track the build scripts, any C compilers that the toolchain builds call out to need to be correctly reflected here, also C++ related stuff is very hot in DMD, and these should also be checked against every C++ related change to DMD.
GDC, LDC, their build scripts will call out to different C++ infrastructure, and these should be correctly checked against those toolchains when doing those builds.
These are very strongly coupled. More strongly coupled than any other C++ code will ever be.
I think these should also be available by default, it's an integral part of our C++ story. At least, it was when it was initially decided.

I understand your argument, and for anything else, I'd agree, but this is intimately linked to the build process of the toolchain. It can't float around on its own.

@TurkeyMan
Copy link
Contributor Author

They're also not like other 'libraries', these aren't subject to active development. They're not 'hot', they'll never change once they're correct, so it doesn't suffer dev characteristics like in-development phobos modules that are being spun up in dub packages.

@TurkeyMan
Copy link
Contributor Author

C++ is definitely not a 'deimos' binding. This isn't 'just some lib' we're making available to D users for convenience... as long as extern(C++) exists in the language, we're making a commitment to get the ABI right, and the stdc++ ABI is a critical part of the compiler vendor's responsibility, and therefore a part of the core language's responsibility to maintain compatibility. There's nowhere the C++ library is distributed separately to the toolchain, and the same applies here.

Consider GDC updates to the next version of GCC, or we upgrade the win builds to VS2019, we need tests to fail, and to correct the ABI in lockstep with toolchain development and release. druntime is the only place for that.

Point is, stdc++ is not a 'lib', it is effectively core language.

@JinShil
Copy link
Contributor

JinShil commented Aug 27, 2018

A properly implemented systems programming language should not depend on C compilers, C++ compilers, or any other language's toolchain; it should only need the linker. The fact that D has a dependency on the C toolchain (at least for now), is a unfortunate drag; please, let's not make it worse.

C++ ABI testing is the responsibility of our CIs. Our CI infrastructure can incorporate a C++ toolchain to verify D's implementation. The CIs could even import the stdc++ bindings from another repository for testing, just as we do when testing Dub packages with each change to DMD.

Furthermore, we don't have the resources for this. @TurkeyMan, if this is important to you, then we need someone like you to take ownership of it, and not just leave it for the rest of us to deal with. We're already way understaffed, and the issues and PRs just keep coming. If it's yours you won't need to wait on or negotiate with anyone to get things done; and you'll be much more productive. Once you have a repository created, we can incorporate it into our CIs like we do the other Dub repositories to ensure DMD remains compatible.


// accepted solution to the C++ namespace problem is to alias symbols outside of the namespace
// https://forum.dlang.org/thread/lppkkcsyvlvhebmlapat@forum.dlang.org
alias array = std.array;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing Ddoc comment.

@TurkeyMan
Copy link
Contributor Author

I really can't see it that way... I would call the opposite; a properly implemented systems language absolutely does have intimate access to the c runtime, it's like the core native lib for any system.

What resources does this require? I'm already carrying this.
If you're talking about ongoing maintenance, that's exactly the point, it is legitimate baggage bourne by extern(C++), they're inseparable. If any language or build/tooling change causes the STL tests to fail, then it's effectively a 'badcodegen' regression, and should be fixed urgently, and it should hopefully not be possible that a toolchain release can happen without a fix.

I expect these are very low-maintenance once they work. This is system code that almost never changes, and in the rare event they do, that's the whole point.

@JinShil
Copy link
Contributor

JinShil commented Aug 27, 2018

@TurkeyMan, I'm not trying to discourage you or anyone else from doing this work; I would love to see this work done, I just don't believe it belongs in the druntime repository. Also I think progress on these bindings would be MUCH more accelerated if they were not here. In fact, putting them here will likely be a hindrance (as you are currently experiencing, and have experienced with your other PRs).

Alternatives include, but are not limited to:

  • A Dub package owned by you or some other group of individual D programmers (This is my current preference).
  • A Deimos repository (better than druntime, but still requires allocating resources from dlang)
  • A separate repository within the dlang organization (though I don't think that is much different than a Deimos repository)

Please note that even if these bindings are in a different repository, they can still be included in the D compiler distribution, if someone believes that would be appropriate for some reason.

I expect these are very low-maintenance once they work

Working on team DMD and druntime is definitely NOT low-maintenance.

What resources does this require?

The maintenance for this work requires creating the bindings (yes, the majority of the work does happen up front, but it doesn't stop there), updating them as needed, responding to questions from users (on the forum, in bugzilla issues, and in PRs), debugging issues, devising solutions to any issues, implementing those solutions as PRs, reviewing those PRs for coding conventions, tests, coverage, documentation, and correct implementation, and negotiating with contributors. All that while trying to keep up with the backlog of issues and PRs we already have competing for our attention.

Furthermore, these bindings require expertise and experience with C++ and stdc++. While most of us probably have some experience there, such skills are not an inherent prerequisite for maintaining druntime. To assume ownership of this, we require active team members that are very familiar with C++ and stdc++. I can't even count on one hand enough team members that would qualify as active let alone having stdc++ experience. Perhaps we could add you to the team as code owner, @TurkeyMan. Are you willing to assume that responsibility, answer questions, handle issues, review PRs, negotiate solutions with contributors, etc.?

Even if we did have you, or a team with the right expertise and resources to create and maintain these bindings, I still don't believe the bindings belong here. Simply put, druntime has a different charter. There are many things in druntime that don't actually belong here. We should be trying to get rid of those things, not add to them.

And where does it stop? We've recently had PRs for adding bindings and version identifiers for the musl C library. Will we eventually need to maintain bindings ulibc++ too? IMO, none of this stuff belongs in druntime. Team druntime should be focused on implementing the D language, and that's all.

If you're talking about ongoing maintenance, that's exactly the point, it is legitimate baggage bourne by extern(C++), they're inseparable. If any language or build/tooling change causes the STL tests to fail, then it's effectively a 'badcodegen' regression, and should be fixed urgently, and it should hopefully not be possible that a toolchain release can happen without a fix.

extern(C++) is useful for any C++ bindings, not just stdc++. I don't believe by having extern(C++) in the language we're obligated to have stdc++ bindings. Even if we were, by someone's subjective judgement, they can still exist in a different repository. We can ensure quality and compatibility by incorporating the external repository in our CI.

With all that said, note that I am just one person on this team expressing my opinion and exercising whatever influence I might have. I do not speak for the whole team; each will have to voice their own opinions. I don't expect anyone to agree with me, but that doesn't mean I shouldn't articulate my point of view. I've already had my say, and maybe even said to much. I defer to those more senior than I for a decision.

"Perfection is attained not when there is nothing left to add, but when there is nothing left to take away." -- Antoine de Saint-Exupery

@jacob-carlborg
Copy link
Contributor

jacob-carlborg commented Aug 27, 2018

If we are going to add bindings for the C++ standard library, does that mean we're going to add bindings for the Objective-C frameworks? Where do we draw the line.

@Geod24
Copy link
Member

Geod24 commented Aug 27, 2018

I have nothing to add that @JinShil haven't said yet.

It's an awesome initiative and I'm looking forward to have it available. I think it should live somewhere special (dlang-community?) and be a dub package, and quickly added to the CI.

@TurkeyMan : Your main concern seems to be about codegen regressions. It is my understanding Buildkite would solve it as efficiently as putting the code in druntime would. Am I missing something ?

@TurkeyMan
Copy link
Contributor Author

TurkeyMan commented Aug 28, 2018

I don't, and don't intend to ever use dub :/

If it's added to the CI, then what's the practical difference from placing it here? It will have the same effect in terms of maintenance.
How do I guarantee that GDC and LDC's test environments also match the DMD configuration? It just sounds like duplicating that work 3 times rather than slotting it in where it's already performed correctly, and requiring no additional work to invoke the CI.
It also sounds like a lot of extra work to bundle a separate repo into releases than to just slot it in here, where it would be automatic.

I still can't really understand the argument. It's about maintenance, but initial development of 3rd party interaction is a lot of additional scripting, and ongoing maintenance are all only harder if it's in some 3rd party location...

We can ensure quality and compatibility by incorporating the external repository in our CI.

If you accept this, then I don't understand how integrating another library parallel to druntime with the same build, CI, and distribution semantics as druntime isn't more work... it just... is more work, for the same outcome no?

Anyway. I yield.
Why did Andrei and Walter put it here in the first place?
Should we remove the existing modules?

@thewilsonator
Copy link
Contributor

Test is no longer run on OSX32, if clean is run rm -rf $GENERATED may fail if its expecting something to have been generated, but that is also easy fixed by the same approach.

@ibuclaw
Copy link
Member

ibuclaw commented Nov 8, 2018

Generally speaking it shouldn't be that difficult to maintain old ports. Solaris 10 and OSX 10.6 will be baselines of gcc 9 for those respective platforms.

There should be better way to manage transitions though. For instance, only supporting one version of FreeBSD is a pain, not having a clear -std=c++XX likewise too. Old platform support should never have a reason for it to stop working.

@thewilsonator
Copy link
Contributor

I agree about the -std=c++XX, but this just disables a test that won't work.

@thewilsonator
Copy link
Contributor

@TurkeyMan please squash this after I'm done with it.

@TurkeyMan
Copy link
Contributor Author

@TurkeyMan

I'm going to bed... I'll waste another day of my life tomorrow!

"I'm going to bed... I'll spend one more day spearheading the important effort of C++ stdlib integration tomorrow!"

Fixed that for ya.

@andralex Thanks man!
Might I suggest taking a serious interest in the productivity issues associated with D contribution from a foundation level? Consider employing a devops professional, and decomissioning/upgrading any build machine that takes >10 minuets. This whole process needs a serious streamlining effort.

@andralex
Copy link
Member

andralex commented Nov 8, 2018

Considering we are. No funds yet.

@andralex andralex merged commit 1bbbc9c into dlang:master Nov 8, 2018
@jacob-carlborg
Copy link
Contributor

I'm currently working as a DevOps, professionally.

@TurkeyMan
Copy link
Contributor Author

TurkeyMan commented Nov 8, 2018

Woah! That was sudden... I would have liked to squash Nick's stream of consciousness (or even reviewed it) ;)

@TurkeyMan TurkeyMan deleted the std_array branch November 8, 2018 19:46
@TurkeyMan
Copy link
Contributor Author

But thanks! That was a grueling marathon!
Now the real fun starts; string/vector.

@wilzbach
Copy link
Member

Fwiw Buildkite is intended to replace the auto-tester, but we currently don't have Windows nor OSX agents connected to it :/

@TurkeyMan
Copy link
Contributor Author

Buildkite... that doesn't even let you see the state of the builds!

@thewilsonator
Copy link
Contributor

See slack.

@jacob-carlborg
Copy link
Contributor

Fwiw Buildkite is intended to replace the auto-tester, but we currently don't have Windows nor OSX agents connected to it :/

What about the existing machines used for the auto-tester?

@wilzbach
Copy link
Member

We don't have access to them.

@jacob-carlborg
Copy link
Contributor

We don't have access to them.

If we know how owns them we could ask.

@wilzbach
Copy link
Member

See e.g. braddr/d-tester#64

@jacob-carlborg
Copy link
Contributor

I don't follow. We don't want to continue using those machines because they're old or physical?

@wilzbach
Copy link
Member

No because we can't install the buildkite agent on them because we don't have access. That was just a link to a previous attempt at improving the hardware setup.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.