Pouring too late can result in crumbly fudge