Emonda ALR rear wheel clearance – is 4mm enough?

I want to get a set of new, modern hoops for my Trek Emonda ALR 2019.

They are 32mm wide at their widest point. I currently run the stock wheels, which are 27mm wide. Thing is, the tightest spot is between the drive-side chainstay and the rim. I currently have 6.5mm of clearance, which would drop to 4mm with the 5mm rim width increase.

Is this enough to be ridden hard safely? There is obviously some flex at the chainstays when putting out power, I wouldn't want to ruin a rim and my frame.

Pics are of the current 6.5mm gap