In a medium sized mixing bowl, combine the pudding mixes and milk together. Blend for about 2 minutes. We used a hand mixer for this, but you could easily use a whisk. Just be sure it's combined well.
Then fold in the Cool Whip to the pudding mixture and mix until well blended.
Lightly spray a 9 x 13 inch pan with non stick cooking spray. Place a layer of whole graham crackers on the bottom of the pan. You may need to break some graham crackers apart to cover the entire bottom of the pan.
Gently spread 1/2 of the pudding mixture over the top of the graham cracker layer. We suggest dropping it in spoonfuls all over the graham crackers and then spreading it around carefully.
Then place another layer of graham crackers over the pudding layer. Piecing it together to cover all of the pudding.
Then add the remaining pudding over the graham crackers following the directions from above by spreading it around gently.
Top that pudding with a final layer of graham cracker sheets.
Place in the fridge while you make the frosting.
Combine the frosting ingredients until smooth and creamy. If you like your frosting thicker, then add a little more powdered sugar. If you like it thinner, then add a teaspoon or two of milk.
Carefully spread the frosting over the top layer of graham crackers until completely covered.
Cover and chill for about 4-6 hours or overnight. The graham crackers will soften up the longer it sets which gives it the soft eclair taste.
This dessert gets better over time. Slice and serve cold.