Lightly coat a 9x13-inch baking dish with nonstick cooking spray and set aside.
Melt 6 tablespoons butter in a large saucepan over low heat.
Add two 10-ounce packages marshmallows and stir until completely melted and smooth.
Remove the saucepan from the heat. Gradually stir in 12 cups Rice Krispies cereal until the cereal is evenly coated with the marshmallow mixture.
Transfer the mixture to the prepared baking dish and gently press it into an even layer. Let the treats cool completely, then cut into 24 rectangles.
Place 4 ounces red candy melts, 4 ounces blue candy melts, and 4 ounces white candy melts into three separate microwave-safe bowls.
Microwave each bowl in 30-second intervals, stirring after each interval, until the candy melts are completely melted and smooth.
Dip 8 Rice Krispie treats halfway into the red candy melts, 8 treats into the blue candy melts, and the remaining 8 treats into the white candy melts. Allow any excess candy coating to drip off.
Place the coated treats on a baking sheet lined with parchment paper or wax paper. Drizzle each treat with the remaining melted candy colors and, if desired, immediately decorate with red, white, and blue star sprinkles.
Let the candy coating set completely at room temperature before serving or storing the treats in an airtight container.