New "Search by Face" App #holocaust #announcements #photographs


Daniel Patt
 

Shalom all,

numberstonames.org lets families of Holocaust survivors and victims "search by face" for loved ones in Holocaust museum photo archives. This began as an effort to find photos of my own grandparents in various museum collections (~70% of individuals in museum photos are unidentified). More than 171,000 faces have been extracted from nearly 34,000 photographs so far. Hope this is a helpful resource, and I'd love to hear about any connections / photo discoveries this helps facilitate. 

Best,
Daniel Patt


Moshe Berman
 

Hey Daniel,

This looks really neat!

What match accuracy would be the threshold for a realistic match? I’m seeing a match score of .60-.70 for non-matches. When I worked at a face-matching startup, the threshold we relied on was mid-upper .90. 

Can your ML engine can be harnessed to also enhance text OCR? Do you have plans to add training buttons to the UI? 

Would it be possible to surface names and locations from USHMM? 


Thanks,
Moshe Berman
Boca Raton, Florida


Shelley Mitchell
 

I tried the site and to my surprise, I found 2 women who looked like my mother. Shelley Mitchell, NYC


Daniel Patt
 

Hey Moshe!

Thanks for your feedback, and for the really great points!

I added some notes below:
What match accuracy would be the threshold for a realistic match? I’m seeing a match score of .60-.70 for non-matches. When I worked at a face-matching startup, the threshold we relied on was mid-upper .90
It's a very interesting question. I think it depends a bit on the use case. i.e., I've seen true positives in the .7ish range and maybe even in the high .6ish range... so that's why I thought it would be helpful to show the raw scores for the top 10 closest matches, and let the reviewer decide. I thought that labeling results as matches / non-matches could potentially bias the reviewer. What do you think though? Maybe for below a certain threshold it should just omit those highly likely true negative results?

Can your ML engine can be harnessed to also enhance text OCR? Do you have plans to add training buttons to the UI? 
Re: OCR, that's a great suggestion! There might be significant content missing / not extracted from these images. As a first pass, we could just add any OCR'd text to some kind of searchable field and also annotate the images. Lots of ways this could go though. Were there some specific use cases you had in mind?

Re: training buttons - yes, that is being worked on as part of a broader moderation effort. 

Would it be possible to surface names and locations from USHMM?

Locations would be quite doable, and we're working on the names aspect of this now.

Thanks again for the thought-provoking feedback!

Best,

Daniel Patt
patt.daniel@...


Shelley Mitchell
 

An interesting observation when I reused the same picture. I received different results with a maximum of .7.  They were less of a resemblance. Any idea what could have caused that?

Shelley Mitchell, NYC 


Daniel Patt
 

Also just as an FYI, an FAQ has been added to the site. Hopefully it helps clarify a few things about what the results mean.

Best,

Daniel Patt 


Shelley Mitchell
 

Tried it again with a younger picture of my mother. Received at least 4 matches of .72 and .71. One already had a name. When you open the original source link, it sometimes gives a name and location of where taken. My next attempt will be to use a picture of my mother’s father. She resembles him. That might box in a family. Similar approach I took for DNA testing to at least determine which side of my parents the match should be from. 


Shelley Mitchell, NYC