How do you find the motivation to develop cntkx?

[faruknane]

Hi @delzac,

Maybe you saw my work. I also develop a library that is similar to cntk in terms of syntax with dynamic graphs. In the beginning, the idea came meaningful to me, creating a deep learning library where users get the most flexible use scenarios. I have developed the library for 9 months almost. I had to change the design of the library 2-3 times. I got tired of those implementation changes. Redesigning is very hard. The cost of mistakes is high. After a while, the idea of creating a library became less interesting and more meaningless. I am about to lose my motivation, to be honest.

How do you keep developing the library that somehow is not popular maybe it will not be ever? How does it make sense to you? And why do you keep going further?

I am solo developing. I contacted with someone but not sure if he will help me with gpu implementations. He says his schedule is so busy and also he wants to help.

Do you happen to know where we can get some support?

[delzac]

Hi @faruknane,

Thanks for take the courageous step forward to seek help. I sorry to hear that you are going through a difficult time developing your deep learning framework. I have visited your github page before and i was very impressed with what you were doing.

To answer your questions, i actually didn't set off to develop cntkx. I was studying deep learning on my own and as part of the learning process, i implement works from research papers. In my early days of using cntk, i found it frustrating that it's hard to find cntk implementation of the latest research. Most of the time, i have to write my own models. So now that i have some code base, why not open source it so that other cntk users do not need to suffer like i did. That was how cntkx was born.

At the start, i did have some hopes that maybe the community can rally around my library and make cntk popular. But half a year later, microsoft decided to stop developing cntk and that really nailed the coffin. Nevertheless, i continue to maintain cntkx since cntkx is simply a byproduct of my work in studying deep learning.

To be clear, i am happy whenever i see people install or clone cntkx. And i do become sad when less or no people do so. But cntkx is not how i want to be useful to the world. I am glad that it is useful to a small group of people. But this is not my life's work, it is a byproduct and that is all it will ever be. In fact, one of the ways i enjoy my free time is to work on cntkx, As that means that i'm reading deep learning papers and implementing them. I enjoy these two things a lot.

If we can go back you what you are facing. I have two observations to offer.

First, what is your reasons for developing the deep learning framework? If you are looking to be useful to the world in this way, and you are not getting traction, i think its normal to feel disappointed. But that is also an important feedback to you. Why are you not getting traction? Does the world need another deep learning framework? What's so special about the framework you are building? You need a good reason if you are going to pull through. For cntkx, not being popular or useful is fine since it is a byproduct of my studying of deep learning.

Second, it seems to me your frustration and increasing lack of motivation also comes from the gap between your current skills and what is necessary to develop the framework. That is to say, you are trying to build a rocket when you currently can only build a car. So you are not seeing progress. Without progress, it is extremely disheartening because the work you put in seems like it is for nothing and you are going nowhere. Are you taking up something larger than you can handle? For cntkx, i enjoy abusing the cntk ops in ways that the cntk maintainers never though of to develop new modules. It is difficult enough to be fun, but not so difficult that it breaks me.

What you are doing is both very impressive and very hard. As a fellow solo developer, i have a lot of respect for you and your work. I hope my sharing brought some clarity for you.

[faruknane]

Yes, now It became more clear why you might continue to maintain cntkx. Whether you publish it or not, you will develop some tools for yourself. Naturally, it's a product of your work and because it is not your main target, you don't get so upset if the library doesn't get popular. I get it.

However, in my case both are opposite. It's directly my work and I want to produce something in deep learning area. The speed of improvement is so high in deep learning that I feel I miss everything while days are passing. Every day, new computing units, new tools, new models, new ideas. I feel choked. So, this time I want to produce something.

What do I do differently than other libraries do? What is the deal?
Well, there are few features that I added but don't know if other libraries have those already, such as supporting multidimensional layers (example: multidimensional RNNs that can solve the unseen word issue with character and word level encodings for word2vec) and layers that can take multiple inputs as layers. My main goal is to create the most flexible library that enables researches in terms of the opportunities it gives.

Do I have the experience and talent to build this library?
I think I might not have enough experience, however, I trust myself in the talent side. Yes, I can build the library, but how will it be? good enough? worth to use? How much will it cost? Those are ambiguous. I believe if I find a few people just 2-3, we can pretty take actions quickly and get speed in building.

[delzac]

Its a big and impressive piece of work you have chosen to undertake, i hope you will pull through! :)