Yep. I should have mentioned that Right clicking listed the same choices as the Open With method. I think I will reinstall Photoshop Elements. I have a feeling that somewhere there is still a folder with an Elements-related file in it that I haven't been able to uninstall.
Have you tried to choose another default to open your JPG's - eg Windows Photo Gallery, which I use to screen my files and do quickie adjustments for family photos? If that works you would be rid of PSE as default and should be able to re-select CS4.