I've posted a couple of Youtube videos. The longest one was about 10 minutes.
I'm not sure what you are asking. Basically, you upload a video and if Youtube accepts it you are fine. If there is some format problem you will get back a message or indication that it failed. Youtube accepts at least MOV, AVI and MPEG formats.
What happens when you post a video is that Youtube's server farm renders the video into Flash. Some quality is lost. So you absolutely don't want to make the Youtube version your archival version.
I am "Mr. I Run My Own Server" with my own VPS running Postfix and Apache, and there's no way I have gotten interested in hosting a streaming video server. The problem will be supporting the bandwidth in bursts. Why would you want to do this? I wouldn't.
As far as encoding or changing formats of videos, products like Adobe Premiere and Microsoft Expression Encoder will do this. The single big feat is transforming AVI (which is uncompressed, semi raw video) into MPEG, Flash, WMV or ASF which are compressed. This is the lengthy, CPU intensive part. My Intel i7 temperature actually rose quite a bit a while back when I compressed one video using Adobe. I kept hearing a fan turning on and off and the four core temperatures were up 10 c or more until it was done.
The MS Expression Encoder is specialized for Microsoft's Media Player formats - WMV and ASF. The Adobe product can produce DVD suitable output (MPEG.)
Lastly, Youtube allows you to restrict viewing to certain users in order to keep some videos private.
I would recommend ignoring all of the video format questions and just try uploading some videos. Note, if the file extension isn't supported by Youtube then it won't work. Only then should you get into re-encoding.