e08d52bb introduced a bug when rotating by 90 or 270 degrees and doing
non-uniform scaling (different scale factor on X and Y axes). The scale
factor was calculated as the ratio between the src box and the rotated
dst box. But scaling is applied after rotation, so the scale factor
should instead be the ratio between rotated src box and dst box.
(cherry picked from commit 34201b0e7f)