So says the American Heritage College Dictionary-
Trifle: 1. Something of little importance of value.
2. A small amount; a jot.
3. A dessert typically consisting of plain or sponge cake soaked in sherry, rum, or brandy and topped with jam and jelly, custard, and whipped cream.
I think that when someone says, "Merely a trifle", they mean the first definition. Usually they are excusing an offense, and assuring the offender that it is of no importance and they are not angry or put out.
Or perhaps they say it when accepting thanks for giving a present. Someone says, "Oh, thankyou! How extragavant you are to give me this!" and the person responds, "Not at all. It is merely a trifle."
Or if it's in response to a question like, "How much sugar would you like in your tea?" and you respond "merely a trifle", then you are saying that you only want a little bit of sugar.
Hope this helps. I've never tried trifle, the dessert, but it sounds good.