Setting up MASM is a fairly simple task. It is the default assembler when you first install RadASM and there are no special considerations. You should use the installer provided by Hutch and install MASM32 first. If you have access to a copy of Win32.hlp, make sure that you copy it into the \MASM32\HELP folder, that is where RadASM will look for it when you hit the F1 key. Install RadASM on the same drive as MASM32 and run it. If you have chosen to install MASM32 to folder other than the default \MASM32, you must alter the paths to reflect these changes, generally only the $A path needs to be changed.
Last time I checked the Win32.hlp file was available from Borland at the url on the links page, it is an 8MB download but well worth the time, no reference will be more valuable to you.