western blot grayscale, 6 lanes, each lane equal width, each band different, first lane 7 bands, second lane 3 bands, other lanes random bands, low noise
AI generation of science images de novo
I have been thinking about this for a long time. At least since the computer in Neuromancer (which I read back when the chip in my computer was a 8088 or 8086) made Cornell-like assemblages.
More recently, as in since 2010 or so, thoughts of what could really be done soon or now, were more constructive; microscopy images as part of -omics. Could AI devour massive quantities of annotated (or even un-annotated) microscopy images, compare, categorize, dissect and sort them, to become a powerful tool for analyzing new images? People collect massive amounts of data, much of it not annotated or shown to anyone, but what if this were fed into a giant database with at least minimal annotation? I would still love to work with computer scientists to make this happen.
Another direction is simulation. What dangers would there be to science if people were able to fabricate images?
We've had discussions of tools that fill in gaps in images. But what about whole images.
An article in the New York Times two days ago spelled out how to do this and rather than send myself an email to do it someday later, I downloaded one of the apps in the article (Midjourney) and tried it.
I started with "western blot." The results were not impressive. Clearly, the system had not been trained with sufficient blots. For instance, the bands are too uniform.. Also, I wonder whether the blots it has been trained with are from publications where they are highly cropped; I expected to see whole lanes. Conclusion: apps for de novo generation of western blots are not a threat yet. But with proper training, this could be a big problem.
After the blots, we will show some microscopy term results, but first the blots.
Text given to create image:
western blot grayscale, 6 lanes, each lane equal width, each band different, first lane 7 bands, second lane 3 bands, other lanes random bands, low noise
western blot grayscale six lanes each lane different
.png)
Obviously, the numbers were ignored and "each lane different" too. I added the additional descriptors because "western blot bands" did not yield anything science-like impressive, although as art I want to see these printed big. Here is a simple western blot bands which looks more like an Arthur Dove landscape (well, maybe better).
I felt the west blot experiment failed, but let's try something standard in our lab but likely obscure for an AI programmed for general use. cell culture scratch assay
The AI tried to get the cell culture part, although it looks more like blood agar plates. Very pretty pictures, but clueless regarding scratch assays. But I pressed on.
cell culture fibroblast scratch assay microscopy phase contrast
These images begin to have the style required for microscopy prizes, but not the reality required. The next attempt was an utter failure. Clearly, not taught yet.
immunofluorescence of HeLa cell, 60X microscope lens, red microtubules, green mitochondria, blue vimentin, black background, white nucleus
The next attempt resulted in some beautiful designs for fiber art. Wow, they are pretty! But clearly the AI is as science illiterate as most artists.
spindle, asters, chromosomes, cell division, microtubules, cell membrane, microscopyspindle, asters, chromosomes, cell division, microtubules, cell membrane, microscopy
I can see people cranking out art. I expect to see "science artists" making splashes with big splashy artworks.
My next attempt went in a differnt direction.
drosophila brain neurons in many colors microscopy
It looks like the AI was fed images of fruit fly brains with neurons mapped. Or at least has refences to brain architecture and knowledge of neuron distribution.their images
The next results weren't quite real enough, but getting very close. Just alittle more training and people will be able to fake images for figures.
first written 2022-10-24 21:45
mcammer@gmail.com