-
Notifications
You must be signed in to change notification settings - Fork 7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
clean up that mess #3
Comments
Lets try that mind map tool: |
Wow...this is pretty old now. Still want to try mindmap? |
Bump with new address... (sorry, don't really how to use this thing yet) https://www.mindmup.com/#m:a1b616f4d096eb013322ac19bf3939d828 |
Are you guys trying to organize the project a bit better? Only thing have so far is maybe putting "aerosols", "mie" and "rayleigh" all in one directory "models" or "theory" or something like that to bundle theoretical/modeling stuff together? |
Did you see the mindmup? Yes, we are trying to reorganize the project. Right now, we don't really have a philosophy regarding the organization. Hagen and I have mostly just discussed it. There is now a branch (NewStructure) of the master where I am going to start rearranging. The repository as it stands is not really safe to fork yet given the terrible condition it is in (too much flux). I think that when
we will be in a good position. Here, we are just discussing the organization. Basically, we want this to be a package that could be registered with conda, pypi or other package distributors. So, to that end, the top level directory for distribution will be |
Gotcha, maybe best thing for me to meet with you guys for lunch as you suggested and get a general feel for the project? |
Sounds good... That being said, I just want to move forward, so if I get time, I am going to move ahead on the branch |
I like your basic structure.
I extended your mindmap a little bit : |
Sorry, this is all over the place and I will likely contradict myself at some point below, but there is a lot to process here. I have a proposal though: if the routine(s) are related to processing data from a non-OTS set of software from an instrument, then THIS SHOULD NOT BE IN THE LIBRARY. In this, I would include non-OTS related instrument routines also such as the POPS. This would include a lot of Hagen stuff such as the Piccolo, Manta Payload, etc, etc. These routines are usually one-off routines and just create a lot of cruft for the user to sort through. On top of this, the routines often contain nothing useful except how to get the data into pandas or some other format. These routines should be the possession of the specific user or group who has need of these. Instrument libraries that should be retained are those that
What say you? btw - you can start to see how some of this is beginning to take shape in the |
What means OTS? |
off the shelf |
Totally aggree. Piccolo has nothing to do with aerosols anyway. We might want to use some of it in the example section to help people to parse there data, in particular when it comes to creating the time-stamps things can get rather tricky. |
Ya...I kind of sorta got into the tools folder and then realized that there was a lot of routines you were using for instrument specific applications. I had started a longer post that commented on these tools, but then ditched it when it became overwhelming. Regarding the tools folder:
|
And regarding the mindmup - I am not sure how I feel about this suggestion. atm in the atmPy implies that this is atmospheric. Wouldn't general atmospheric routines such as lapse rate, etc. just go in the main folder? |
Regarding tools: Regarding the atmosphere folder: |
You might be right about this. Maybe we need a "general" folder? Regarding plotting and pandas - do we want to lock the user into a particular way of expressing the data returned by the routines? pandas might be good but your plotting routines assume that the user will use matplotlib as the plotting tools; is that what you want? We might want to go over this folder in person together. Don't get rid of anything on your end with the idea that you are restructuring - this could create some merge headaches down the line. Just let me know what you definitely can get rid of and I will. |
pandas is essential to the sizedistribution module. Everyone should use pandas anyway ;-) Locking people to matplotlib ... good question. However, these plotting functions are just soooooo useful. I would prefer not to get rid of them ... at least not all. |
Yeah...but they are essential to you. Is the average user going to find them essential? Regarding pandas - I think using it is fine. I just think the routine you have in there is not useful to everyone or is easily reproduced in private libraries... |
I think we are mixing things up here, lets talk about this another time. Maybe we are getting to much into the detail here for now. I think we have an idea about the general Structure and maybe should go ahead and reorganize things into the folder. We can then focus on content of particular modules. What do you think? |
I think that is incorrect - when thinking about structure you are inherently thinking about organization. We are attempting to make the library more coherent. To that end, we need to figure out 1) what belongs and 2) where it belongs. If these plotting and pandas routines are essential to the function of your stuff, then we need to figure out what stuff they are essential to so that we can place them exactly where they need to be. Stuffing everything into a generic |
|
Hmmmm....I thought I replied to this, but apparently I didn't. I will address these points randomly:
This gets to a point I made above - if you don't expect users to use them, then why are they there? This library is not for developers but for users, no?
My thought is that things that are intended for use by others shouldn't be "messy". If it is messy, then the directory hasn't been given enough thought.
I don't understand this. Why do you need them? What do they do that is integral to the functionality we are trying to provide? |
They are there because they are used by modules that are used by users.
They are also used by modules that are used by users. Almost all data in modules written by me is based on pandas DataFrames and Panels, so I came across similar issues in different modules. Do reduce redundancy I am collecting these tools and reuse them. |
So, this is what is messed up with the tools folder. I just discovered that you are duping some of my code. You have |
sure thing! |
was just checking out the the new structure branch. I know you are not a fan of very descriptive names and I am a fan of over descriptive names. What about a compromise? in aerosols:
The last couple of dayes I worked on simplifying sizedistribution and actually splitting it up and making some of the class attributes available out side the class, so it can be used in a more functional form (as you suggested). I am fare from ready though, so it is probably still rather confusing. |
Hmmmm....I haven't checked in for some time...I am testing some things to see if they make sense. I will take a look at this again. Regarding Don't try to do anything related to this branch - I think that it is going to be tough merging things. I am trying to refactor appropriately where I can. |
I suggest that because I have the feeling this topic will come up more often in the future if we don't come up with a rule ;-)
I think we should merge more rather than less, this makes us more familiar with how to handle problems. Also, I want to go ahead with things and would like to avoid to big clashes. |
The pythonic mantra is readability. |
So silly compromise solution, instead of instrument or instru what if you called instru "device" instead. It has readability, but still maintains a clear tie to the contents? |
sounds good to me! |
This statement clashes with every repo. When is the last time you saw "utilities" or "library" or "input-output" written out in a module name. Take a look at naming conventions in popular repos. From PEP-0008 on **module names:
From Python Coding Conventions general comments about naming:
and example provided
Long names can (and do) obstruct readability |
The Nevertheless, this is a discussion for another issue that was called out by Hagen not for the structure but simply for naming conventions. I suggest that this discussion be shifted to another issue that focuses on these. Right now, I am more concerned with the structure as well as testing than I am about naming. I think naming will become more of an issue as we swing around and start examining what the package actually contains. For now, I am going to leave the naming argument... |
I knew you would pick the these abbreviation which had been put into stone a thousand years ago and which are used in every programming language. If we come accross these things we of course do not write them in full length! But as you quote further down. There is now reason to abbreviate: user_profile, menu_options, word_definitions which are longer then instrument or sizedistribution |
ok! If its about the structure only, why not finalizing the mindmap and putting it into action?!? |
Last thing to say about naming: you have chosen three of the bad examples provided. These are overly vague names and even if they weren't, there would not be any meaningful way to shorten them. I would suggest that things like num and conc and atm are meaningful to parties that use these packages. Surely you didn't struggle with what the library |
What do you mean finalizing the mindmup? I am hoping that with |
But how can I implement my changes if you don't want me to merge? |
I thought, if the mindmap is set, it takes an hour to just do the changes. |
I suggest we do multiple steps.
|
Maybe if you want to do this then we tear the whole thing down and start adding everything back in. What you suggest above sounds...exhausting... I am pretty certain that what you suggest is going to take about a year. The refactor tool in PyCharms makes moving a breeze. Why do we need to go file by file? Are you still changing your stuff that is in the library now? |
I would use the refractor for each of these steps, thats why I thing its not problem to do it in the way I suggested it. In particular step 1) would be a no brainer. And yes I am still working on the libraries almost every day. If you want I can take my current working branch and reorganize it according to our proposed folder structure this weekend. Since nobody else is using our stuff anyway, I can than just pull it back into master and we both start testing our stuff and start fixing thinks. We are in a particularly nice situation right now, since our stuff is not too much entangled. So you can fix your stuff and I fix my stuff. Once we are done with that, we start talking about the next wave of changes. What do you think? |
OK. I am now working off master and you will see an update in another 10 min. Pull. No code was deleted or changed but a lot was moved. |
…ssue #3. No code deleted. Just moved. Files tagged for removal binned in for_removal directory in root.
Done. Added a folder that contains files that I think should be removed. You may disagree. Let's discuss. Also, will be adding tests. Let me know if we can close this. |
what ever. |
please just change instr to instruments and phys to physics and we then close it. |
and rad to radiation |
everything is lower case what about making General to general |
I did. Did you not see it? Feel free to change the Can you get rid of |
I am closing this. I think now it is time to get to the details. |
Everyone! :-)
give some suggestions about how to rearrange the repository!!!!
The text was updated successfully, but these errors were encountered: