Should JOSS publish papers on code that relies on proprietary software/programming environments?

[labarba]

We haven't discussed the limits on scope for the journal regarding code submissions that rely on a proprietary library or runtime.

What if the submitted code is itself open source, but it relies on a runtime that is closed and a paid product? (The typical example is MATLAB code.) It can be argued that even though the submitted code is open source, it does not integrate with an open-source software stack, thereby inflicting a barrier on users.

It is my personal opinion that such code submissions are not really open (and we should not publish them). We recently had a similar discussion in the NumFOCUS board, and decided that such projects would not be eligible for fiscal sponsorship under the NumFOCUS umbrella.

What do other editors think?

[cMadan]

I think it depends on the availability of the paid product. In the case of MATLAB, most relevant educational institutions provide access via an institutional license or in specific computer labs (along with first-party toolboxes). If the code required a less common paid product, I can agree with not accepting them at JOSS. However, this does bring up the fuzzy issue of how to judge if the required product is 'common' or not.

As an an editor, I do think that using MATLAB is acceptable, but I am biased in that MATLAB is my main programming language. (And biased in that this issue originated from review of a submission of mine.)

[arfon]

What do other editors think?

I favour a slightly more lenient approach here: I think relying upon a proprietary runtime is acceptable as long as it is possible for the reviewer to complete the review process:

[ ] Installation: Does installation proceed as outlined in the documentation?
[ ] Functionality: Have the functional claims of the software been confirmed?
[ ] Performance: Have any performance claims of the software been confirmed?

While I would rather see everyone using open source languages I think we should recognize that this is not a realistic option for many researchers in the near-term.

It also occurs to me that encouraging contributions to the open source ecosystem (as JOSS papers) in these communities that rely upon closed-source/proprietary runtime environments can only serve to elevate the role of open source in the long term.

[labarba]

most relevant educational institutions provide access via an institutional license

Do you have data to back that up?

And what about non-educational institutions? Should we call open software that is only available in (select) educational institutions?

It is true that The Mathworks have done aggressive marketing in universities, which is reflected in the institutional access and cheap licenses for students. This availability disappears when students graduate, and this is a problem in my eyes.

[pjotrp]

I am against it because it prohibits people from running the code. Including reviewers. Also there are people in countries that don't have the money for these tools. @biorelated what is your view here?

[danielskatz]

I'm also mildly against it.

On Jun 27, 2016, at 12:51, Pjotr Prins <notifications@github.commailto:notifications@github.com> wrote:

I am against it because it prohibits people from running the code. Including reviewers. Also there are people in countries that don't have the money for these tools. @biorelatedhttps://github.com/biorelated what is your view here?

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHubhttps://github.com//issues/142#issuecomment-228804891, or mute the threadhttps://github.com/notifications/unsubscribe/ACx2NTqeQR-9rAylF5rVnSMd6-JpEueZks5qP_9kgaJpZM4I_R4T.

[kyleniemeyer]

I second @danielskatz's "mild" opposition (sorry @cMadan 😦)

I don't think we need to put a blanket limit against, e.g., MATLAB code, because an open-source interpreter does exist in that case (Octave). However, not all MATLAB code will run in Octave without some work, particularly if it depends on proprietary libraries.

@arfon In response to your requirements for the review process, what happens when someone best qualified to review doesn't have access to the runtime or libraries? Or if someone submits an open-source software package that relies on something less "common" than MATLAB, but is otherwise similar to the current situation in question? I imagine we wouldn't accept something that none of the reviewers could install or run, even if the submitter (and possibly certain other institutions) did have access. Therefore, MATLAB (in the current example) would in a sense be given a pass simply because of its popularity.

I absolutely agree that we want to encourage contributions to the open-source ecosystem—which I think JOSS does through its very existence—but don't we also want to encourage a fully open software stack? And, therefore, discourage one that isn't fully open?

(OK, through the course of writing that, I think my opposition escalated slightly.)

[pjotrp]

Free and Open Source Software is just that. JOSS is for FOSS. When there is a requirement for non-free software we should not accept that because it prevents people from using and auditing the software. Even a platform as Windows or Microsoft Excel is not acceptable. FOSS software has to be made available for other platforms for everyone to use.

I may sound extreme, but I am actually pragmatic. In the rich world we live in a cocoon when we think all the resources are 'free' to us. 80% of the students, teachers and researchers are not able to Matlab or SPSS because of its costs. So, software built on that belongs in the rich world. It does not belong in a journal that promotes FOSS.

[cMadan]

While free 'dependencies' would definitely be preferable, I think the bigger issue is if JOSS is built around open science practices or FOSS. I was excited when JOSS launched as it provided a venue for publishing my analysis code as toolboxes that others can use and modify in their own work, while also counting as being 'published' in a journal. However, if this is insufficient and JOSS also requires that all dependencies be free and open, this is a very important distinction.

There are ideals, but there are also practicalities--I can't program in other languages nearly as well as I can in MATLAB, so I won't be re-writing my code to Python for the sake of open science, I would just be ostracized from publishing it via JOSS. Additionally, the particular submission that brought up this issue requires a freely available third-party MATLAB toolbox. I don't know for sure if that works in Octave, but if it does not, the point goes further in that even though free (GPLv2), anything using that code would also be not publishable at JOSS without re-writing the code into another language.

If no reviewers are able to run the submitted code, I agree that it couldn't be accepted at JOSS. It clearly would be preferable if no paid products were dependencies for submissions, but I feel it is too extreme to enforce this as a journal policy--effectively discouraging open science practices unless an individual adopts another language.

[pjotrp]

At the moment we sit between writing software for science that is available to all (my view) and publishing papers on free software that sits on dependencies that cost money. I understand @arfon's point that we should encourge people to publish their code thereby hopefully encouraging them to change their ways. @cMadan I am sure you don't mean to really say that you won't change your ways just because you won't learn Python so as to share you great gift with others ;).

My suggestion is as follows: we should make it a point to state clearly that we prefer and encourage FOSS software that does not depend on proprietary closed source tools, and certainly not on tools that cost money for mentioned reasons (limited audience, hampered auditing). Meanwhile we create a loop hole that we can allow such contributions, provided the authors help the reviewers run the tools - i.e., by providing access to such tools. We can not ask JOSS or reviewers to buy the licenses.

How does that sound? I think that is pretty fair. I know of at least one paper in another journal that got rejected because the reviewers could not run matlab. Why would be be different? We need to be able to audit the software.

[george-githinji]

One can make an argument that work derived from proprietary environment should automatically get disqualified. But again what is the role of a journal? JOSS is a journal, and as a journal it has a mandate to welcome, share and make knowledge accessible. As long as the author is willing to provide the full source code and disclose proprietary pieces, then his work should be accepted and published. If the work meets and passes the review criteria and as long as the proprietary licence does not restrict the author from providing his code or works derived from it under a FOSS license, it should be acceptable for publication. I agree that some of the proprietary runtimes may not be accessible to people with less income, but I feel that is a separate issue.

[pjotrp]

Thanks @biorelated.To add one more thing, I am less principled than I may sound (though I don't think we should underestimate the importance of free software). I actually champion the idea of a journal that accepts all comers (provided the software is FOSS, there is some scientific contribution and the coding is non-trivial). I strongly believe that papers, even blogs, that are great, simply drift to the top and get cited.

The rest drops off the radar. Out of the 100K+ scientific papers that get published annually, how many are actually relevant?

That is closer to PLOS ONE. Great software begets great papers. JOSS can accentuate FOSS and ask people in review to opt for a free platform the next time. The reviewers should be able to run the software. If we publish 1,000 paper per year, I am sure there will be quite a few great ones.

[Kevin-Mattheus-Moerman]

I agree that reliance on proprietary components should be minimized and we should fly the "free and open-source flag" as high as we can. However, in reality I think we cannot completely avoid codes relying on some proprietary components.

Software works relying on proprietary components should be allowed as long as they follow the OSI definitions of open-source (which includes software relying on proprietary programming environments) and there are reviewers available.

That reviewers need to be available is of course true for all software in general. I.e. there may be various exotic and rare free programming languages for which we may never have reviewers. Or programming languages that have gone out of fashion.

A "proprietary component" can be a proprietary programming environment (e.g. MATLAB, LabView, Mathematica), another closed-source software package (e.g. I know of people developing Python code to create input files for, and process output from, closed-source analysis software) or even hardware (e.g. control software for robotic devices, control software for MRI scanners). Again, I feel as longs as there are reviewers available, and the software submissions themselves follow OSI definitions, I think they are suitable for JOSS.

Side notes/tangents:
1)
Out of interest. Are most FOSS projects actually free and open-source "all the way down" (as in "turtles all the way down")? Or would one hit a wall at some built-in code or a set of ones and zeros one could not openly or humanly read?

@labarba touched upon fiscal sponsorship of projects. This is a different discussion but I actually feel special funding should be made available to help people switch to free and open-source tools. Currently at all of the institutions I worked with or for, a campus wide MATLAB license was available. So in that sense MATLAB has always appeared free to me and my colleagues. I am not saying this is a good thing. This probably comes out of our research money in some automated way and we cannot opt-out. It also removes any incentive from me and my peers to even try to switch to stuff like Python (I would love to though). I mainly work with MATLAB and I've calculated that it would take me at least 3 months to switch all my skills and tools from MATLAB to Python (while probably hardly advancing the research I should be doing). As a post doctoral researcher I simply cannot invest that amount of time in translating to another language to do the same thing I am already doing. Agreed it would be more open but my superiors would see it as a waste of time. Unless we could actually opt-out of these licenses e.g. as a department or research group. Alternatively there should be dedicated funding available to help research groups switch to open and free tools. The additional funding would help bridge the sluggish translation period. Once the open-source tools are in place it could self amplify in a department, e.g. I'd be teaching new research students Python instead of MATLAB.

[bachmeil]

I recently had a coauthor send a program that runs on Matlab (but not Octave). I cannot currently run that software because Matlab is not installed on my machine. Even if I had my department pay for a Matlab installation for this one project, I would only be able to work on the paper in my office.

I don't understand the connection to "open source" in that case except as a marketing tool. It's true that this isn't an issue for well-funded research universities. I just don't see the connection to open source.

[oheim]

Compared to a classic journal the JOSS paper with proprietary dependencies would still be superior, since the published software itself is free and you could study and migrate it. This is much more than I can expect from common scientific software nowadays. Of course there'd be a practical problem to review it, but you'd have similar problems if I proposed a software which has other exotic dependencies (like it only operates in a power plant, in a submarine, in space, …).

IMO, there is no need to accept such a paper. If the software shall be free, it is easy for the author to get rid of proprietary dependencies nowadays. For example, if it is written in the popular Matlab language, it should be easy to get it to run in Octave with little effort. To request this extra effort from the author would benefit the software and in turn the users.

There might be corner cases where you have no replacement for some non-free dependency. So I'd say “no” to non-free dependencies unless one has a very good reason.