The colormunki, in my experience, is terrible for determining LCD brightness based on ambient light level.
Actually for the luminance level of the display manual adjustement is the best way... no measurment device knows what you are seeing...
Am I crazy or going about this the entirely wrong way?
well, why not... if you have no other measurment device.
To adjust the luminance level of the display to about 120cd/qm set the light meter to spot, set ISO200, f8 (1/2) and 1/15'' and measure 10cm in front of the display. Adjust the luminance of the display to match the light meter. Better than nothing maybe. But again... the luminance level should be "pleasant" to you not to the device. If you have a viewing booth with some ISO standard lights to eye up prints you should try to match the luminance level accurate (but again manually).