render: resize(): don't overflow the number of scrollback lines

The config system allows setting the scrollback lines to
2**32-1.

However, the total number of grid lines is the scrollback lines plus
the window size, and then rounded *up* to the nearest power of two.

Furthermore, the number of rows is represented with a plain 'int'
throughout the code base.

The largest positive integer that fits in an int is 2**31-1. That
however, is not a power of two.

The largest positive integer, that also is a power of two, that fits
in an int is 2**30, or 1073741824.

Ideally, we'd just cast the line count to a 64-bit integer, and call
__builtin_clzl{,l}() on it, and then take the smallest value of that,
or 2**30. But, for some reason, __builtin_clzl(), and
__builtin_clzll() appears to ignore bits above 32, despite they being
typed to long and long long. Bug?

Instead, ensure we never call __builtin_clz() on anything larger than
2**30.

Closes #1828
This commit is contained in:
Daniel Eklöf 2024-09-09 06:51:10 +02:00
parent 8a4bbbf5cb
commit 1925593a37
No known key found for this signature in database
GPG key ID: 5BBD4992C116573F
2 changed files with 34 additions and 4 deletions

View file

@ -93,6 +93,11 @@
* Some invalid UTF-8 strings passing the validity check when setting
the window title, triggering a Wayland protocol error which then
caused foot to shutdown.
* "Too large" values for `scrollback.lines` causing an integer
overflow, resulting in either visual glitches, crashes, or both
([#1828][1828]).
[1828]: https://codeberg.org/dnkl/foot/issues/1828
### Security