How can I increase user mode virtual address space to max 3GB?

You have to increase the user-mode virtual address space using BCDEDIT command as follows:

BCDEDIT /set increaseuserva <Megabytes>

If you want to increase it to 3GB: BCDEDIT /set increaseuserva 3072.

http://msdn.microsoft.com/en-us/library/ff542202.aspx