Combining stable diffusion with semantic search to categorise, tag, and query 100k images of hot dogs

Follow the full discussion on Reddit.
I have been particularly interested in generating synthetic datasets using stable diffusion for various machine learning purposes (and I also think this is going to be a big area). However, I started to run into problems trying to manage them or even know what was in them (since there is a large variance in the outputs for the same prompt). I think one compelling solution to this problem is using a semantic search system to query, store, and categorize the generated images. I did some experimentation (see below) to see how this could work on a synthetic dataset of 100k hot dogs. One other thing that really dawned on me as well was the progress with prompt engineering could really impact search and search query curation due to the shared models (i.e. CLIP). Anyway, the exploration is below and I would love to hear any feedback!