3D Scanning Techniques and Workflows

[morbo]

Nice one! I'm convinced! ;)

[gjpetch]

Wow, I'd be curious to see what you got with an active projected noise pattern! I find that agisoft loves having some kind of splotchy detail, and hates smooth shiny surfaces (like skin), I've had promising results projecting noise patterns like this:

[Infinite]

Wow, I'd be curious to see what you got with an active projected noise pattern! I find that agisoft loves having some kind of splotchy detail, and hates smooth shiny surfaces (like skin), I've had promising results projecting noise patterns like this:

I can certainly try but I honestly think it wont make a difference. I was seeing the same effects with SK. There is a weird cut off between, sharpness of focus, lighting, closeness too subject (detail) and DOF. Sometimes noise can help a little, sometimes it makes it worse! I've got bunch of cameras in close at split levels so they receive allot of high detail which seems to overcome the need for noise.

Problem with using an Active pattern is it means a need for a 2nd set of data to process which really isn't practical as it doubles the amount of data you have to record and handle.

[piek]

It's amazing how often it works right off the bat, and for the price...:thumbsup:

[colorina]

Love this thread <3.
I am so impressed with your work Infinite!

I have also been experimenting with this technique and Agisoft photoscan.

My budget is not very large, so I have tried a little cheaper approach.
I have 6 Canon powershot A495 that I have put CHDK in, to be able to use with a remote trigger that I have made from an old pocket lamp.

My boyfriend volunteered to model for this :-).
What parameters do you find best when you build the geometry i Photoscan?

http://i391.photobucket.com/albums/oo354/auvesst/printscreen4.jpg

http://i391.photobucket.com/albums/oo354/auvesst/printscreen3.jpg

[Infinite]

Love this thread <3.
I am so impressed with your work Infinite!

I have also been experimenting with this technique and Agisoft photoscan.

My budget is not very large, so I have tried a little cheaper approach.
I have 6 Canon powershot A495 that I have put CHDK in, to be able to use with a remote trigger that I have made from an old pocket lamp.

My boyfriend volunteered to model for this :-).
What parameters do you find best when you build the geometry i Photoscan?


Thanks Carolina : ) very nice results. Your approach is very innovative and cool :)

I just use pretty much the stock settings. To Align I do High and Generic.

For Build Geometry I use Arbitrary, High, Smooth, 1000000 (maybe 250,000) 0.5 and 0.1.

I hope that helps. I find a multi Core system REALLY reduces processing times. 8 Core if you can.

[colorina]

Thanks for you reply, I will try with the same settings :-)

I will also experiment a little with the angles between the cameras. Maybe you have some good advices on that too?

[Infinite]

The best thing I can suggest is more is better. A friend of mine did some tests with a Canon5D MKII he took over 30 images per person over about 1 minute, capturing a 180 Arc. Agisoft loved all those and the results were amazing, super detail to beat any current system on the market but took about 2 hours to process. I then tested with just 9 images and the results were great also, the same with 4. I would say more Cameras the better but your 6 should be good. Agisoft loves filler Cameras and Pairs of images, by filler I mean inbetween shots, try and sync with a flash if you can.

You need the cameras to fire first say on 1/4, then a small delay on flash to sync the sensors.

[colorina]

Thank you!

[misha]

...besides the name being impossible to remember, and not having the program inherent on my machine, these are good results... my friend, Fred, who by the way is repping ZCorp, shot these photos...

[colorina]

I made a quick try with only one camera (Canon 500D) walking around taking pictures. It was a bit noisier, but easier to perform. It has to be worked on, but it is a fun way to start a sculpt I think.
http://i391.photobucket.com/albums/oo354/auvesst/printscreen8.jpg

[Infinite]

Interesting results Carolina, a great way to start off a sculpt : )

[mala]

Hi Everyone,

Here's a headscan i've recently made:

http://img593.imageshack.us/img593/2448/scansetd.png

http://img708.imageshack.us/img708/2001/setdmeshtest2flatcolour.jpg

http://img687.imageshack.us/img687/8576/setdmeshtest1.jpg

Photos taken with 3 Canon450D inside my LED lighting Dome,basic mesh build done with AgiSoft Photoscan then fine detail added using a world space normal extracted from sequence of lighting states in the dome.
The detail created by the normal map is geometry not a "render time effect".

I haven't done any 'clean up' on the mesh from Photoscan other than a quick smooth,hence my wonky eyes and lumpy forehead.

Rendered in vrayforc4d....i need to spend more time on shader+render/light...just used the globally illuminated set of photos from the dome for the colour map(hair and all) slapped into the vraysss2 material..no other maps in shader yet.

I hope to pass the photos/data over to Lee in the next few weeks as i'm sure he can produce some better renders than my crappy attempts.

Cheers,
mala

[colorina]

Thank you Infinite :-). I will bring in the other cameras as well in future tests.

That is a really good result Mala! Your own lighting dome? Cool, how does it work? Can we see pictures?

[mala]

Thanks Carolina,

The photos for this scan i took in my 2nd prototype dome,there is some video of my first prototype on Lee's Blog (http://www.ir-ltd.net/led-array-by-merry-hodgkinson) and a bit more info here (http://www.ir-ltd.net/light-array-hdr-probes)

My second version is a bit smaller and the construction is a little different..i have re-programmed the lighting states to give a proper gradient now also.
Cheers,
mala

[moonjam]

That is absolutely incredible mala... I've just completed our first commercial project for which we were able the use Photoscan & I wish we had results as clean/detailed as yours!

Wonderful.

[popol]

Really good result mala and nice rendering too!

we're currently building a mini lightstage but we have some problem to wire all the LED. :(
What type of board do you use to drive all your led ?

[morbo]

Wow Mala...thats pretty impressive!!!

so is that extracted from 3 pictures? or from a 3 camera rig and multiple/simultaneous shots?
Did you use any filters on your lenses?

the resolution is amazing! I'd be curious to see what your results look like without the LED light dome and if they're as crisp. Do you have any pics?

Thanks for sharing...very inspirational!

[Infinite]

Very cool Merry! When I'm back I would love to check out the data.

[mala]

Thanks All.

@Morbo the basic mesh build in photoscan is from 3 photos,but the detail is extracted from a series of photos taken with the same 3 fixed cameras.
This all happens quickly..full set of photos taken in about 1.2-2.8sec...depending on the number required.
This can happen faster..the only limit is the maximum continous frame/shot rate of the cameras...i only have 450Ds and they will only do 3.5fps max.

No filters used on the cameras and i'll post some pics of the basic mesh build with photoscan later.

@Popol...yep wiring something like this up is not fun!....i have 156 LEDs in my rig..so it took quite some time.Sorry i'm not going to reveal much info on the control system/board.All i'll say is that is different to the lightstage and other versions control systems...i can control many other things with it as well.

@Lee....hope you're having fun mate ;-)...give me a shout when you're back,as i said i'm sure you can make some much nicer renders with this data.

Cheers,
mala

[mala]

Here's some pics of mesh the straight out of photoscan and after i did quite a brutal smooth on it to remove the noise,then i applied the normal map to add the fine detail.

http://img829.imageshack.us/img829/7861/setdmeshes.jpg

As you can see my smooth was a bit OTT..i've lost a lot of form around the eyes/eyelids....but this was a very quick test..and i will take more care with other meshes.
I could probably have pushed the detail with the normal map a little further..and some better focusing+higher res photos would also make this detail sharper of course.

Afraid i don't have any versions that I made without my LED dome...i think in this thread somewhere are some examples of my first tests with Photoscan though...in general i would say there is more noise in those examples.

The advantages of the LED dome other than being able to extract this normal map for the fine detail is that you get a 'globally illuminated' colour map with no self shadowing and very little specular/reflections for your skin texture,which pretty much needs no touch up work. As well as other images of different lighting directions that are great as reference for sculpting and shader creation.

Cheers,
mala

[NextDesign]

Here's a scan that I've been working on for a while now; trying to get as much detail out of it as possible. One 5D MK II, two photos, no calibration.

http://i24.photobucket.com/albums/c30/NextDesign/passiveScannerResult.png

The detail created by the normal map is geometry not a "render time effect".

Interesting. I remember Lee looking for something to do this. Did you do this in Zbrush? I'm not aware of any way you can displace a normal map onto a piece of geometry. Care to elaborate a bit?

[mala]

Hi John,

That's a very nice result for only 2 photos, shows what a difference higher resolution makes.Wish i could afford some 5Ds...6 would be nice Santa..

What software did you use,photoscan?

For the normal map displace of the geometry...Mmm..well i guess i can say a bit,as Lee has already mentioned it on his blog ....I use Cinema 4D.

Its about the only advantage to using C4D compared to you guys using Max/Maya..at least Max/Maya seems to get a more updated and better version of Vray.

The developer of Vrayforc4d disputes it but i'm sure something is different with the SSS2 in our Vray compared to Max/Maya.

Cheers,
mala

[NextDesign]

For the normal map displace of the geometry...Mmm..well i guess i can say a bit,as Lee has already mentioned it on his blog ....I use Cinema 4D.

So by using the MoGraph displacement deformer?

[morbo]

Happy New Year all!

Thanks for the reply Merry! (sorry mine is so late...eek! holidays :) ) I'm pretty amazed at the results you've shown! Thanks for posting some of your WIP pics! helpful.

I really have to make myself an LED dome...I agree...having an instantly available diffuse map, and a Normal map are pretty sweet!

John...that looks pretty amazing!...especially for just two photos.
You said you've been working on the scan for a while now...do you mean processing the mesh? or just reshooting/processing until you got a result that looked good. That looks ridiculously crisp. Can you share what lens, settings you used?

[NextDesign]

John...that looks pretty amazing!...especially for just two photos.
You said you've been working on the scan for a while now...do you mean processing the mesh? or just reshooting/processing until you got a result that looked good. That looks ridiculously crisp. Can you share what lens, settings you used?

Hey Morbo, thanks!

I've been working on the same images, trying to get the best results from them. I took these photos outside, while facing the sun; late in the afternoon to get the light straight in my face. I shot the photos at f/16 with a 1/320 second exposure. It took many photos to get a set that worked, as the sun was setting very quickly; which made it necessary to shoot the stereo pair in quick succession. I think the contrast from the low sun actually helped grab more detail, as I've been having difficulties getting the same results using a flash or overcast lighting.

I've written my own software to get even more detail, but at the moment I don't have a machine that can handle a 26 million poly (quad) mesh :D

[Infinite]

Very cool research Merry!

John: Some interesting results. Would be great to see without the embossing or Highpass filter on the raw mesh.


I've written my own software to get even more detail, but at the moment I don't have a machine that can handle a 26 million poly (quad) mesh :D

26 Million sounds ridiculously high. I can honestly say on a face like that 500,000 - 1 Million Raw output is ample enough.

The only way a 26 million polygon mesh would be needed would be if you had some kind of micron scanning device to pick up pore level. 1-2 Million is sufficient enough to pick up basic pore level detail, before embossing any kind of bump, then maybe Subdivide up to 6 Million or so but no more is needed really.

Here are some examples of 500,000 to 1 Million Polygon Models. Even the Disney data is below 1 Million.

Disney original Data and the same 8 images processed in Agisoft.

15 minutes to process with a Highpass emboss. Disney use their own "mesoscopic" emboss which is a hallucinated Highpass trick really, just a little better than the PS one.

http://www.ir-ltd.net/uploads/cgfeedback/agi-disney-test-01.jpg

These were taken by a friend of mine Alexander Tomchuk, who used a Canon5D MKII, flash set and took about 30 images per face around the subject. 2 Hours processing time, out at 750,000 polygons from Agi. No Highpass emboss, raw from Agi.

http://www.ir-ltd.net/uploads/cgfeedback/AGI-01a.jpg

http://www.ir-ltd.net/uploads/cgfeedback/AGI-02a.jpg

http://www.ir-ltd.net/uploads/cgfeedback/AGI-03a.jpg

[mala]

Hi Lee,

Cheers mate...i will send you over some of the original photos...only just remembered.

The AGI/Disney comparison ...what was the res of the original photos?

Also interesting to see Alexanders scans,this is the first time i've seen them at this larger size...there is not quite as much detail as i first thought but they are very sharp!

Cheers,
Merry

[Infinite]

Hi Lee,

Cheers mate...i will send you over some of the original photos...only just remembered.

The AGI/Disney comparison ...what was the res of the original photos?

Also interesting to see Alexanders scans,this is the first time i've seen them at this larger size...there is not quite as much detail as i first thought but they are very sharp!

Cheers,
Merry

The Disney photos were around 16MP - 21MP.

With Alexanders, because of the 21MP resolution the High Frequency details if added using the HP hack come out quite nice. They were taken over 2 minutes, so I would imagine allot of movement had crept in but not bad for 1 Camera, mobile.

[NextDesign]

John: Some interesting results. Would be great to see without the embossing or Highpass filter on the raw mesh.

Hi Lee, here's a side-by-side of the simplified-raw mesh and a smoothed version.

http://i24.photobucket.com/albums/c30/NextDesign/passiveScannerResultSxS.png

26 million is indeed high, but that would retain all the data from the images. I would then (if I could) run this through a quadratic-edge decimation algorithm to retain that detail.

I may have to rewrite some of my custom software to decimate the mesh itself.

[Infinite]

Hi Lee, here's a side-by-side of the simplified-raw mesh and a smoothed version.

http://i24.photobucket.com/albums/c30/NextDesign/passiveScannerResultSxS.png

26 million is indeed high, but that would retain all the data from the images. I would then (if I could) run this through a quadratic-edge decimation algorithm to retain that detail.

I may have to rewrite some of my custom software to decimate the mesh itself.

Sure, I'm just trying to save you some headaches with working on 26 million polygon meshes. From your screen shots there is no evidence that it is doing anything. Try and and stick to around 1-2 million polygons, maybe 6 if you have to. It's much easier to handle.

The only reason to go that high would be if you had some kind of super accurate, high frame rate structured light capture system, combined with Normal Map Displacement.

Try a test with a 2 Million Polygon Model output and a 26 Million Polygon Output :)

[morbo]

a lil' OT, but pretty cool, and supportive of any performance capture posts....


http://www.youtube.com/watch?v=EuGFYyXJ10A&feature=player_embedded#!
http://www.youtube.com/watch?v=3R2K5yXpWIM&feature=player_embedded#!

markerless facial motion capture... using tech from http://dynamixyz.com

[{bukkit}]

WIP portrait using photofly (123D catch)

http://img89.imageshack.us/img89/9349/phtfl.jpg

http://img856.imageshack.us/img856/714/modelw.jpg
raw data

http://img191.imageshack.us/img191/9141/zbrush.jpg
only minor cleanup

http://img440.imageshack.us/img440/5714/nrmz.jpg
just a little normal map test

http://img836.imageshack.us/img836/4629/wirefit.jpghttp://img205.imageshack.us/img205/5937/nosk.jpg
retopo done in topogun. sits on the face like a glove

imho great results for a easy&free solution :)
next im gonna try to do an ear scan with it, if it fails im gonna give agisoft a try, some great agisoft results in this thread

[{bukkit}]

update :
http://img580.imageshack.us/img580/1181/scanittothelimitlimiiii.jpg
http://img220.imageshack.us/img220/1181/scanittothelimitlimiiii.jpg
photofly + details taken from texture

[NextDesign]

Very nice Lukáš, what camera did you use/how many photos?

[j3st3r]

Amazing thread guys.

Anyone could suggest some settings to achieve good result with Agisoft/123Catch?

Cheers

Jester

[{bukkit}]

@NextDesign thank you. it was shot with eos 550D (50mm lens). not sure how many photos were finaly used, there were a lot of trial and error :) but i think that more than 50, less then 80

@j3st3r
not sure if its ok to post links from cgsociety here, but here is my wip thread about this project, you can find there some (i hope) usefull tips

http://forums.cgsociety.org/showthread.php?f=43&t=1019262

[j3st3r]

Thanks, Bukkit, great link
I don't agree with the topology especially for facial animation, but the overall look is EXCELLENT. Thanks again!!!

[{bukkit}]

any suggestions on what to change (regarding the topology) ?

[j3st3r]

Actually the best I found was the Avatar topology (I don't have the image here) and also there were a good example of Digic Pictures for their first Mass Effect3 intro.

I think if you make a search on cgtalk for facial topology you'll find it.

The base is I think that the topology follows the skin folds. If you make grimaces you'll see how your skin folds. Make these folds your topology guidelines and your ready to make a great topo. Your topo supports wrinkles, and folds almost only from displacements, while the other supports the folding by polygons, what you can enhance with displacement. Both way is good, but I pefer the other :)

[{bukkit}]

i was planning to combine the two :) + influence object to suport the frame of the scull.

http://img.imgur.com/wLW0o.jpg

[misha]

scanned with reconstructme, textured in zbrush....using preview and flat renders

http://www.cgfeedback.com/cgfeedback/picture.php?albumid=54&pictureid=892

http://www.cgfeedback.com/cgfeedback/picture.php?albumid=54&pictureid=891

Mike

[gjpetch]

I've been trying agisoft photoscan + polarising filters to remove specular, + blotchy random facepaint (so that agisoft has something to register against), pretty good results, though I'm still getting some small scale noise. I think I'm possibly running up against the limitations of my crappy camera.

[misha]

pretty good results....full body next....

let me know what you think...

http://www.cgfeedback.com/cgfeedback/picture.php?albumid=54&pictureid=895

http://www.cgfeedback.com/cgfeedback/picture.php?albumid=54&pictureid=894

http://www.cgfeedback.com/cgfeedback/picture.php?albumid=54&pictureid=893



Cheers,
Mike

[gjpetch]

Another test, again using the high pass filter bump map technique for micro details. I did two scans, one used blotchy random facepaint and polarizing filters to remove the specular, the other used no facepaint and polarizing filters turned 90 degrees to enhance the specular. And then I aligned the scans using meshlab. Basically the scan without specular gave a nice clean detailed mesh, and the textures from the other scan gave me the bump map. The downside is that any differences in pose or facial expression between the two scans will mess things up.

[misha]

.... very nice....

Cheers,
Mike

[gjpetch]

Hey misha, I wonder if you've seen this link, about hacking better closeup results out of the kinect by giving it glasses!
https://groups.google.com/group/reconstructme/browse_thread/thread/eca392cfb4a508ad#

[misha]

Hi Greg,

Yeah, the color scans are using the hires technique, with 2.5 reading glasses... pretty cool hack....)

...by the way, I posted on that link....

Cheers,
Mike

[{bukkit}]

http://img.imgur.com/TOw4F.jpg

http://img.imgur.com/r09nr.jpg

[NextDesign]

The advantages of the LED dome other than being able to extract this normal map for the fine detail is that you get a 'globally illuminated' colour map with no self shadowing and very little specular/reflections for your skin texture,which pretty much needs no touch up work.

Hi Mala, I'm interested in how you're generated the normal map. Are you following the method proposed by Ma et al 2007; or are you following something like "Normal Map Photography" by Ryan Clark? (http://zarria.net/nrmphoto/nrmphoto.html)

[mala]

Hi John,

Yes I'm pretty much following the Ma et al method.

Cheers,
mala