License question

[JarbasAl]

Piper is licensed as MIT

Espeak-ng is licensed as GPL

https://github.com/rhasspy/piper/blob/master/src/cpp/phonemize.hpp#L13 <- doesnt this mean MIT license is invalid, piper needs to also be GPL

[synesthesiam]

Piper depends on an espeak-ng fork, which is available under GPL here: https://github.com/rhasspy/espeak-ng

[JarbasAl]

IANAL but my question is exactly that, Piper depends on GPL so it can't be licensed as MIT, having a GPL dependency (non-optional) means you must keep the license

[synesthesiam]

I'm not a lawyer either, so I don't see the point in speculating. Like any piece of modern software, users of Piper will need to evaluate the licenses of all its dependencies and consult the appropriate legal counsel. Pull most any package from PyPI and you will be in the same situation.

If I'm ever advised by a relevant lawyer to change the license, I definitely will.

[JarbasAl]

well, i would not consider this speculation as this kinda is the entire point of GPL, as a developer it is you responsibility to comply with the license, users aren't affected by this directly and do not need to evaluate licenses as they wont be breaking any license clause by just being users

in OVOS we have automations to ensure license complicance, so the pypi issue you mention is something I am deeply familiar with, accidents happen, pretending there is no issue is a whole another thing once it is flagged to you

I opened this issue out of concern for the project, as this is currently my favorite TTS.

a quote from gnu.org GPL FAQ

A system incorporating a GPL-covered program is an extended version of that program. 
The GPL says that any extended version of the program must be released under the GPL if it is released at all. 

thanks for looking into this

[synesthesiam]

I believe the point of the GPL is to prevent companies from using open source software without contributing back to the community. Everything I create is open for others to view and modify, and I encourage upstream contributions.

I say "I believe" because, again, I'm not a lawyer and I'm not qualified to interpret legal code. If someone from the espeak-ng project actually has an issue and would like to chat about it, I'm all for it. I appreciate the concern, but I think we can all be good open source citizens without the threat of legal boogeymen.

[ChanceNCounter]

Came to my attention because I’m the OVOS license compat nerd, and this analysis includes our Piper plugin in addition to Piper itself (heritable licensing is a big headache)

The threat of legal boogeymen in this situation is exactly why Steve Penrod moved MycroftAI away from copyleft, and it’s what warned us off those Mycroft components that were relicensed in the Dinkum era.

OVOS has gratefully ported a whole bunch of that work, but not the copyleft stuff, because we also prefer the MIT and BSD licenses, and we don’t want to inherit the copyleft restrictions. We have bots digging for missed hard deps, license policy compat checks in our CI. This stuff keeps me up at night.

The “good” news is that rendering the dependency optional would probably be sufficient. I haven’t actually sunk my teeth in yet, but that’s the usual situation. Throw in a bit of API that would make it possible to import some other lib in its place, and a way to make that configurable, and you’re (probably, again, haven’t dug in) GPL compliant.

[JarbasAl]

I'm not gonna rehash the old debate of permissive vs copyleft licenses, but any search around those keywords will show this is not some boggeyman i'm making up... complying in spirit is not complying with the license. Countless internet debates show you this is a real concern. But my intention is not to discuss merits of either, just to point out the issue

"i didnt mean too" is not a good argument, but if your stance is you don't care until a lawyer comes up there's not much else I can say....

With that said, I will no longer be using the license as an argument to recommend piper over mimic3 like I have been doing and will point to this discussion when the subject does come up

[jamesmyatt]

I haven't investigated the exact situation, but I expect that the following are true:

• Piper is not a derivative of espeak-ng: it's only dynamically linked
• https://github.com/rhasspy/espeak-ng is a derivative of espeak-ng: it's modified code

GPL only requires derivatives to be GPL-licensed, so piper can choose its own licence, while https://github.com/rhasspy/espeak-ng cannot. If espeak-ng used AGPL, then it would be a different matter.

[JarbasAl]

I haven't investigated the exact situation, but I expect that the following are true:

* Piper is _not_ a derivative of espeak-ng: it's only dynamically linked

* [rhasspy/espeak-ng](https://github.com/rhasspy/espeak-ng) _is_ a derivative of espeak-ng: it's modified code

GPL only requires derivatives to be GPL-licensed, so piper can choose its own licence, while rhasspy/espeak-ng cannot. If espeak-ng used AGPL, then it would be a different matter.

the dynamically linked argument would be valid for LGPL only, not for GPLv3

it is precisely point 1 that is the concern, if the user is able to replace this lib with a non-GPL equivalent then there wouldnt be an issue as espeak would just be a plugin among countless others, just empowering users to change it

as is, users can not install this without espeak so it is not optional

[ChanceNCounter]

The question with respect to Piper, and I’m still commenting pre-analysis, would be if it’s a combined work, not whether it’s derivative. The static vs. dynamic linking distinction hasn’t been properly settled in court; rms and the FSF contend that it’s all the same, and it all amounts to a combined work.

Philosophically and logically, I disagree. As a representative of a nonprofit on a shoestring budget, I can’t afford to argue with the FSF’s legal theories about its own licenses.

[jamesmyatt]

I understand that lots of people have differing opinions on this (which is only of the big issues with GPL), but I believe the most widely held is that only static linking with libraries creates a GPL'd work. The dynamic linking, especially of the kind that Python does, is generally allowed.

[JarbasAl]

I understand that lots of people have differing opinions on this (which is only of the big issues with GPL), but I believe the most widely held is that only static linking with libraries creates a GPL'd work. The dynamic linking, especially of the kind that Python does, is generally allowed.

in regards to python, I couldn't articulate the concerns better than Steve Penrod did https://softwareengineering.stackexchange.com/questions/119436/what-does-gpl-with-classpath-exception-mean-in-practice/326325#326325

the link above pretty much explains why we avoid even LGPL in python libs under OpenVoiceOS

EDIT: glad to see this issue reopened instead of swept under the rug

[synesthesiam]

As a representative of a nonprofit on a shoestring budget, I can’t afford to argue with the FSF’s legal theories about its own licenses.

This is what I mean by "boogeymen". Why on Earth would the FSF come after Piper or OVOS for GPL violations? The legal teeth of the GPL aren't meant for us.

I'm not saying we can just ignore software licenses, but even within this tiny thread we already have multiple interpretations of GPL 😄
Our stuff is open (the whole point of open source), so I'll leave the wording of a text file to lawyers.

The “good” news is that rendering the dependency optional would probably be sufficient.

Then I've got more good news, as this is the case with the next version of Piper (in a side branch)! Raw text phonemes will be supported, and espeak-ng is only invoked (via dynamic linking) if the voice model requires it.

[JarbasAl]

Then I've got more good news, as this is the case with the next version of Piper (in a side branch)! Raw text phonemes will be supported, and espeak-ng is only invoked (via dynamic linking) if the voice model requires it.

great! this should solve the issue completely

[jamesmyatt]

It's up to the relevant IP owners to enforce the terms of the licence they choose. In this context, the only relevant IP that FSF owns is the copyright to the text of GPL v3. It's not FSF's responsibility (or right) to enforce GPL terms of projects that use GPL.

So the real question is what owners of the GPL'd IP chain in espeak-ng want to do.

But I understand if you prefer to just avoid the uncertainty of GPL.

[synesthesiam]

My main concern is being overly restrictive when I don't need to be. From this thread in the espeak-ng project you can read:

As we are not able to contact the original developer of eSpeak (Johnathan Duddington), it is not possible to modify the license for the existing codebase, including adding exceptions.

They also state in the same thread:

If you don't want to open code of your application, technologically easiest and legally correct way is to use unmodified, stand-alone eSpeak NG through its CLI interface. Calling its unmodified API from separate thread/process can also be legally correct.

To me, this communicates all I need to know:

1. No one actually understands the GPL license
2. The espeak-ng code owners are fine with API use via dynamic linking for closed source programs, so Piper is fine

[ChanceNCounter]

The basic concern is the unlikely, but ever-present possibility these factors could combine:

1. Anyone who contributed GPLed code upstream could have a gripe
2. The FSF will provide pro bono lawyers to C&D GPL violations on request
3. FOSS is political, and resolving license disputes is time consuming. It doesn’t seem like it should be, but it’s a decent way to ruin a rival’s weekend
4. FOSS is political, and “X violates copyleft” can buy you a lot of misery all by itself

Plus, we don’t have attorneys of our own. If we ever were afoul of a copyleft clause, we wouldn’t even have stalling tactics at our disposal. We’d just have to drop everything and sort it out.

None of this should be part of a programmer’s world. I resent Stallman every day.

[JarbasAl]

I keep seeing dynamic linking mentioned here, can someone explain to me why LGPL was made if dynamic linking is allowed under regular GPL? what is the point of LGPL then?

The Lesser General Public License (LGPL) was created to address some of the concerns related to the usage of the GNU General Public License (GPL) in certain scenarios, particularly when it comes to libraries.

The primary difference between the LGPL and the GPL is in how they handle the distribution and usage of the licensed software when it is linked with other software.

The GPL requires that any program that is combined with GPL-licensed code (whether statically or dynamically linked) must be distributed under the terms of the GPL, effectively making the entire work subject to the GPL's copyleft requirements. This can be restrictive for libraries because it may force developers to license their entire applications under the GPL if they want to use a GPL-licensed library.

In contrast, the LGPL includes a special provision, the "LGPL Linking Exception." This exception allows developers to use LGPL-licensed libraries in proprietary or non-GPL compatible applications without triggering the copyleft requirements of the GPL. Specifically, when a program dynamically links with an LGPL-licensed library, it can be distributed under different terms (proprietary or open source) without being subject to the copyleft requirements of the GPL.

The LGPL strikes a balance between the desire to encourage the use and improvement of free software while still allowing for the creation of proprietary software that can utilize LGPL-licensed libraries.

Espeak is GPLv3, not LGPL

If i am wrong in the above please correct me for my own education

EDIT:

the FSF recommends devs assign copyright to them, espeak seems to have done that

so the FSF can enforce the license, espeak authors need not be involved at all i think

[ChanceNCounter]

There are two ways to inherit the GPL: by creating a derivative work, such as by forking a GPL project, or by creating a combined work, such as by incorporating a GPL project into your own.

Static vs. dynamic linking is brought up with respect to the GPL in reference to derivative works, the untested (but sensible) theory being that C code which calls a DLL is not derived from that DLL.

The FSF and Stallman claim it’s a moot point, based on the (also untested) theory that by calling the library, you’re creating a combined work.

The LGPL’s linking exceptions get around the FSF’s own position on combined works, and was created exactly for this reason. But then we run into the additional problem where it uses language that doesn’t apply to Python, and that’s why Mycroft didn’t historically tolerate the LGPL, either.

[jamesmyatt]

FSF only requires that contributors to programs that are already FSF-copyrighted assign the copyright of those contributions to FSF. This in necessary to ensure that FSF maintains the copyright to the entire source code of programs they own. It makes no sense for all GPL'd projects to assign their copyright to FSF, since then they'd lose control of their own projects.

The copyright notice you linked to refers to the text of the GPL licence, not the espeak-ng source code.

LGPL differs from GPL by explicitly allowing both static and dynamic linking without creating a derived work. As others have said, many people hold the view that GPL also allows dynamic linking without creating a derived work, but not static linking.

[JarbasAl]

thanks for the explanation!

many people hold the view that GPL also allows dynamic linking without creating a derived work, but not static linking.

this was the missing piece of the puzzle for me

[Qubrick]

1. https://en.wikipedia.org/wiki/GNU_General_Public_License#Linking_and_derived_works
2. https://opensource.stackexchange.com/questions/8797/what-if-i-dont-know-whether-my-program-will-be-linked-to-a-gpl-library-or-not
3. https://opensource.stackexchange.com/questions/1188/what-are-the-arguments-for-considering-dynamic-links-not-to-constitute-derivativ

It's not clear about dynamic linking and definition of derivative work. People and organizations have different opinions. The best solution is to remove espeak entirely so no one have to deal with GPL anymore.

According to this, using espeak via CLI is the only way around if you don't want to remove espeak entirely and keep permissive license. Last option is to change the license to GPL-3.0 which kind of eliminate purpose of having this project under MIT license in the first place.

[XthemeCore]

Could you please keep Piper under MIT license? I really want to use this in my game but now I'm nervous because of its dependencies licenses.

Is there anyway to replace espeak with something else? I read on OpenVoiceOS mentioned issue seems like gruut would help but you said that it has num2words as dependency? And I found some epeak references in gruut repository as well.

[synesthesiam]

I do not plan to change the license of Piper. I personally do not believe that dynamic linking and calling an API necessarily produces a derivative work (there are obvious degenerate cases, of course).

If legally advised to change the license, I would choose to drop espeak-ng as a dependency instead. But here is why I don't want to do that: the open source text to speech community is being held back because we are fractured and solving the same problems over and over. Grapheme to phoneme (g2p) conversion is a very complex problem, involving language-specific pronunciation dictionaries [1], grammatical knowledge, number/date/time parsing rules, etc.

I wrote gruut to avoid using espeak-ng originally. It works, but suffers from the same problems as most every other open source g2p solution: works well for one human language (typically English) and one programming language (typically Python). Not a great solution for Piper, which is focused on many human languages and is intended to be usable from C++/Rust/Python/etc. Gruut also inherits all of the licensing ambiguities mentioned in [1].

<soapbox>
If open source text to speech is ever going to be more than a toy for languages other than English, we need to rally around a common base. epseak-ng could be that base if we can put to rest the questions about licensing.
</soapbox>

[1] The licenses of these pronunciation dictionaries is almost always swept under the rug! The CMU dictionary, for example, has an open but non-standard license. How about the license on an AI model like DeepPhonemizer? It's MIT, but uses WikiPron which is Apache 2. WikiPron in turn uses Wiktionary, whose contributions are dual licensed under CC-BY-SA and GFDL. What does that imply for any downstream projects of DeepPhonemizer? No clue! 😆

[JarbasAl]

Like the patent troll boogeyman cost Mycroft it's business, any ill intentioned third party can bring the GPL boogeyman to a FOSS project they don't like (because it's direct competition or because they are bored). The cost of the legal battle is enough to ruin a project even if you are technically right (like mycroft was).

I'd love for a precedent to be set and prove you right! that would be the best thing in FOSS so we wouldn't need to worry about this stuff and could get back to coding

The fear of said legal battle is also enough to hurt adoption, companies won't want to risk that battle and can just pick a different technology. Not something most users care about but enough to make (at least some) smart gadgets in the wild not ship Piper as their default TTS. Not that the chance was high to start with, but this makes it orders of magnitude lower

[mikejgray]

As a user and sometimes-dabbler in the guts of this stuff, I can appreciate keeping it simple and the community coalescing on a single set of tooling to both avoid rework and ensure forward progress. Piper's currently my favorite TTS option, edging out Coqui. While it's small enough now not to attract unwanted attention from bad actors, I'd hate to see that change for the worse.

You raise some good points about the licensing of pronunciation dictionaries and ML models. I think the primary difference there is that those licenses are largely an open question, untried in court, while there is some established precedent with GPL and LGPL. I'm not a lawyer and don't play one on TV, but I am wary where legal precedent has already been laid out.

What's the most productive path forward? Addressing licensing questions around espeak-ng, as you suggested?

[synesthesiam]

One path forward (not necessarily the most productive) might be to work with the OVOS/Mycroft/Rhasspy communities to port pieces of Lingua Franca and gruut to a new set of C++ or Rust libraries (with good Python bindings).

What is really needed is a collection of libraries that build on each other, but let you use them independently. Coqui TTS and Piper (next version) support using text as "phonemes" for a model. Great, no phonemizer needed! But now you will not get numbers, times, and dates pronounced correctly because no dataset can contain all variations. The libraries we need are:

1. Word and sentence boundary detection in text
   • Could be rule based and/or machine learning based
2. Converting numbers to words
   • Needs to handle cardinals, ordinals, years, etc. as well as grammatical inflections
   • Must detect ambiguities like "1/2" being "one half" or "January 2nd" or "February 1st"
3. Converting dates and times to words
   • Needs to handle multiple formats and grammatical inflections
   • Must detect the same ambiguities as numbers library
4. Detecting and expanding initialisms/acronyms
   • This requires a language-specific database in many cases -- there's just no way to know that "NASA" is not pronounced as individual letters while "NASM" is.
5. Splitting apart words that are made of pronounceable parts, like usernames
   • @JarbasAl is a great example -- a g2p model will do much better with "Jarbas" "A" "I" split out
6. A word pronunciation database
   • The phonemes must be consistent across languages and over time for any downstream models trained on them
   • Pronunciation in some languages depends on grammatical features, such as part of speech or gender
   • Users should be able to add to or override pronunciations
7. A grapheme to phoneme guessing model
   • This is a last resort after the pronunciation database, and only after numbers/etc. have been "wordified" and words have been split apart (step 5)

Lastly, there are some language-specific things that need to have for text pre-processing and phoneme post-processing. For example, Arabic and Persian need diacritics added back in to the text, and French phonemes need to blended according to liaison rules. Not sure if those things should be in their own library or added into one of the existing ones.

Should be easy, right? 😆

[ChanceNCounter]

Well… there is a perspective from which the introspection was the hardest part with LF, and that was a Python problem. Once we figured out a way for developers to drop in their localizations, the pace picked up considerably.

Crowdsourcing the work is a moderate challenge. Crowdsourcing the code review…

[Qubrick]

Seem like piper now support raw text phoneme via phonemize_codepoints() with DEFAULT_PHONEME_ID_MAP in piper-phonemize
Thank you for all of your hard work, @synesthesiam.

However, it still no phoemizer. According to @synesthesiam's suggestion, the solution is to port gruut to C++. Not sure if Cython and Nuitka would help.

[ChanceNCounter]

The OVOS Foundation would be happy to assist with a Rust port in the form of yours truly, but it’ll be harder for us to furnish C++ devs. Between our core programmers, we’re conversant, but not remotely fluent.

[synesthesiam]

That would be awesome! I've only stuck with C++ because I haven't taken the time to learn Rust.

[ChanceNCounter]

That would be awesome! I've only stuck with C++ because I haven't taken the time to learn Rust.

That was reason enough to spin up a chat room for Rust for Mycroft stuff. You'd be more than welcome! Like I said to the OVOS folks in the room,

I'm hoping to pull folks from the other projects in here since we're probably going to share some of the libs we rewrite first

Everybody's gotta learn Rust at the same time, to rewrite most of the same code, and I imagine most will have the same sorts of questions coming from a heavy Python background

Right now, it shows up under the OVOS space, but if it needs to serve the ecosystem without the OVOS brand, it's easy to republish Matrix rooms under additional names.