It is a well known problem when using caffenol, any of the recipes of caffenol.
If you are using a coffee based developer, you might have had this problem too, on your B&W pictures. It may depend on the film used, but with Ilford HP5 for instance it is a very annoying thing, look to the following picture, scanned at 1200 dpi:
You can make the picture more presentable via software. I used the GIMP filter 'despeckle' and I've got the following:
It is said that if the film is very well washed after the developer, the spots will be tolerable, but I washed very well after the developer and it was still untolerable.
On my modest opinion, coffee developers are not compatible with the Ammonium Thiosulfate fixer. If coffee is a must for you, as it is for me, then better use another fixer. Sodium Thiosulfate is the best alternative, or even Ammonia. Sodium Thiosulfate may be prepared by boiling a solution of Sodium Sulfite with Sulphur.
The next I will be using Sodium Thiosulfate that takes longer to fix but, I hope, will give clean negatives. The spots look like silver grains and I make no idea why they precipitate on the film developed with coffee. The same fixer was used with Parodinal and no spots were produced.