You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This isn't intended as any kind of specific request of feedback, I was just hoping to possibly provide a source of creative inspiration.
It's important to provide a fairly standard navigation interface that users are already familiar with so that they can figure out the basics -- I'm not arguing for changing any of that, however if we never experiment with alternative approaches, it's hard to solve the problems that users may struggle with using existing options.
I think the basics work pretty well for curating a small collection of photos -- it gets harder when you scale up the size of your collection though.
Search and tagging seem to be the standard "state of the art" when it comes to addressing larger scale collections -- in contrast to the previous standard which involved manually curating related files into folders or albums. Photoprism already supports both approaches and attempts to replace some of the manual work of tagging photos with available image recognition models (a big assist for larger scale collections).
Unfortunately, search has a big limitation -- it pre-supposes that the user knows what they're looking for. Even if you get around the technical hurdles of indexing all of your content and providing an easy to use query interface, search has always focused on the problem of finding that which you're looking for rather than exploring that which you do not necessarily know.
The tag line Browse your life suggests a goal of enabling browsing and exploration rather than simple search. I just imported about 20k photos (25k listed in originals, 17k listed in photos which I think means about 8k were identified as duplicates?). I don't really want to scroll through all 17 thousand of those hopefually unique images from start to finish in one go. One thing I've noticed is that every time I come back, I have to start over at the beginning again -- a "jump to" option would probably be nice -- particularly in the "group by similar" sort -- it would be nice to see images similar to a specific starting point of my choosing rather than having to scroll through the entire collection. But again, that's a bit more of a search problem.
Last year I was working on a completely different, but somewhat related problem -- I wanted to build a tool that would help me figure out what my wife wanted to eat. I treated it like a classic search problem with constraints:
given a multi-dimensional space of food characteristics, treat a persons current appetite as a search in that space for the food that most closely matches the criteria of their appetite
assume that, while the user may not know what they are hungry for, they can at least say which of two options they would prefer, and treat the target appetite as being food that is closer to the option that they would prefer than the one that they would not prefer
use some sort of gradient descent or set of bisections to quickly converge on the food that best matches the users appetite
All I had to do was build a metric space for the food -- foods that are more similar should be closer to each other than foods that are less similar. I considered using one of the intermediate layers of a pretrained model like nasnet as one of the inputs for building my "food space", but to start with I decided to go with a tag based approach -- I thought it would make it easier to edit the space in the beginning. It helped that most of the image sources had already tagged the images, so I had a decent starting point. I ended up using fasttext pretrained word vectors to map each of the tags onto a 300 dimension word space, then averaged the vectors for each of the tags applied to a food image to map that food image onto the space. (I went a step further and ran PCA on the points to reduce teh 300 dimensions down to the 12 most significant projections).
I didn't have a good quantitative way to measure if anything I was doing made sense though -- I mostly just wanted to visualize what was going on and qualitatively see if it seemed like it made sense, so I created an interactive scatter plot that showed all of the food points on two of the 12 dimensions at a time. The UI isn't pretty, but if you hover over a point you can see the food and the food details, and if you click on one of the values X1 - X12 on either of the X or Y axes, the plot will rotate to project the points on that dimension.
That helped me 1) qualitatively convince myself that similar foods were showing up together, 2) identify clusters of similar food and 3) identify outliers which usually needed a bit of manual metadata cleanup.
It mostly worked for me, but my wife and another friend of ours were helping to curate the image descriptions and tags and it just didn't really make sense to them. It was too foreign to them -- they needed a UI more like the gallery views that they were used to with infinite scroll... They also wanted to be able to quickly and consistently add tags with dropdowns and chiclet style drag and drop, plus be able to filter to see all of the foods with a specific tag, and similar foods, so I threw together a basic metadata editor with a gallery view.
They really didn't care about the 12 dimensions, but I was still trying to explain the point of them, so I added in fingerprint barcharts to try a different way of showing where each food showed up in the food space and the ability to sort foods by each of the 12 dimensions (I attempted to name the dimensions, but did a bad job and never updated them after resetting the projections...)
Again, I'm not suggesting you do any of this, but I thought you might want to take a look just to draw some inspiration.
Like I said earlier, I really like the idea of grouping by similar, but in a really large collection, the current photoprism UI doesn't really help me navigate to any specific set of region that I'd like to look at -- I just have to scroll from some arbitrary starting point until I get to a set of similar photos I'm interested in. It would be nice if there was a way to summarize the full space and then zoom in on an area of interest -- kind of like the geospatial map view works, but instead plotted onto a more abstract space with some random sampling of thumbnails displayed to give an idea of what pictures I might find in the various regions. Maybe that's something people would like, but then again maybe it's just too foreign of an interface.
Either way, I hope this helps give you some "food for thought" ;-)
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
This isn't intended as any kind of specific request of feedback, I was just hoping to possibly provide a source of creative inspiration.
It's important to provide a fairly standard navigation interface that users are already familiar with so that they can figure out the basics -- I'm not arguing for changing any of that, however if we never experiment with alternative approaches, it's hard to solve the problems that users may struggle with using existing options.
I think the basics work pretty well for curating a small collection of photos -- it gets harder when you scale up the size of your collection though.
Search and tagging seem to be the standard "state of the art" when it comes to addressing larger scale collections -- in contrast to the previous standard which involved manually curating related files into folders or albums. Photoprism already supports both approaches and attempts to replace some of the manual work of tagging photos with available image recognition models (a big assist for larger scale collections).
Unfortunately, search has a big limitation -- it pre-supposes that the user knows what they're looking for. Even if you get around the technical hurdles of indexing all of your content and providing an easy to use query interface, search has always focused on the problem of finding that which you're looking for rather than exploring that which you do not necessarily know.
The tag line Browse your life suggests a goal of enabling browsing and exploration rather than simple search. I just imported about 20k photos (25k listed in originals, 17k listed in photos which I think means about 8k were identified as duplicates?). I don't really want to scroll through all 17 thousand of those hopefually unique images from start to finish in one go. One thing I've noticed is that every time I come back, I have to start over at the beginning again -- a "jump to" option would probably be nice -- particularly in the "group by similar" sort -- it would be nice to see images similar to a specific starting point of my choosing rather than having to scroll through the entire collection. But again, that's a bit more of a search problem.
Last year I was working on a completely different, but somewhat related problem -- I wanted to build a tool that would help me figure out what my wife wanted to eat. I treated it like a classic search problem with constraints:
All I had to do was build a metric space for the food -- foods that are more similar should be closer to each other than foods that are less similar. I considered using one of the intermediate layers of a pretrained model like nasnet as one of the inputs for building my "food space", but to start with I decided to go with a tag based approach -- I thought it would make it easier to edit the space in the beginning. It helped that most of the image sources had already tagged the images, so I had a decent starting point. I ended up using fasttext pretrained word vectors to map each of the tags onto a 300 dimension word space, then averaged the vectors for each of the tags applied to a food image to map that food image onto the space. (I went a step further and ran PCA on the points to reduce teh 300 dimensions down to the 12 most significant projections).
I didn't have a good quantitative way to measure if anything I was doing made sense though -- I mostly just wanted to visualize what was going on and qualitatively see if it seemed like it made sense, so I created an interactive scatter plot that showed all of the food points on two of the 12 dimensions at a time. The UI isn't pretty, but if you hover over a point you can see the food and the food details, and if you click on one of the values
X1
-X12
on either of the X or Y axes, the plot will rotate to project the points on that dimension.That helped me 1) qualitatively convince myself that similar foods were showing up together, 2) identify clusters of similar food and 3) identify outliers which usually needed a bit of manual metadata cleanup.
It mostly worked for me, but my wife and another friend of ours were helping to curate the image descriptions and tags and it just didn't really make sense to them. It was too foreign to them -- they needed a UI more like the gallery views that they were used to with infinite scroll... They also wanted to be able to quickly and consistently add tags with dropdowns and chiclet style drag and drop, plus be able to filter to see all of the foods with a specific tag, and similar foods, so I threw together a basic metadata editor with a gallery view.
They really didn't care about the 12 dimensions, but I was still trying to explain the point of them, so I added in fingerprint barcharts to try a different way of showing where each food showed up in the food space and the ability to sort foods by each of the 12 dimensions (I attempted to name the dimensions, but did a bad job and never updated them after resetting the projections...)
Again, I'm not suggesting you do any of this, but I thought you might want to take a look just to draw some inspiration.
Like I said earlier, I really like the idea of grouping by similar, but in a really large collection, the current photoprism UI doesn't really help me navigate to any specific set of region that I'd like to look at -- I just have to scroll from some arbitrary starting point until I get to a set of similar photos I'm interested in. It would be nice if there was a way to summarize the full space and then zoom in on an area of interest -- kind of like the geospatial map view works, but instead plotted onto a more abstract space with some random sampling of thumbnails displayed to give an idea of what pictures I might find in the various regions. Maybe that's something people would like, but then again maybe it's just too foreign of an interface.
Either way, I hope this helps give you some "food for thought" ;-)
Beta Was this translation helpful? Give feedback.
All reactions