I wouldn't call the claim that Lewis may have had relations with Mrs. Moore "baseless." You may need to do a little more research of your own.
We've discussed this issue many times here at the Wardrobe.
Here's a couple of links to those threads.
Personally, I don't believe that Lewis did have Mrs. Moore as his mistress, but there are strong arguments to the contrary. I guess we'll never really know for sure.