Before this patch, when the cell dimensions changed (i.e. when the
font size changes), sixel images were either removed (the new cell
dimensions are smaller than the old), or simply kept at their original
size (new cell dimensions are larger).
With this patch, sixels are instead resized. This means a
sixel *always* occupies the same number of rows and columns,
regardless of how much the font size is changed.
This is done by maintaining two sets of image data and pixman images,
as well as their dimensions. These two sets are the new ‘original’ and
‘scaled’ members of the sixel struct.
The "top-level" pixman image pointer, and the ‘width’ and ‘height’
members either point to the "original", or the "scaled" version.
They are invalidated as soon as the cell dimensions change. They, and
the ‘scaled’ image is updated on-demand (when we need to render a
sixel).
Note that the ‘scaled’ image is always NULL when the current cell
dimensions matches the ones used when emitting the sixel (to save
run-time memory).
Closes#1383
Images with an aspect ratio of 1:1 are by far the most common (though
not the default).
It makes a lot of sense, performance wise, to special case
them.
Specifically, the sixel_add() function benefits greatly from this, as
it is the inner most, most heavily executed function when parsing a
sixel image.
sixel_add_many() also benefits, since allows us to drop a
multiplication. Since sixel_add_many() always called first (no other
call sites call sixel_add() directly), this has a noticeable effect on
performance.
Another thing that helps (though not as much), and not specifically
with AR 1:1 images, is special casing DECGRI a bit.
Up until now, it simply updated the current sixel parameter value. The
problem is that the default parameter value is 0. But, a value of 0
should be treated as 1. By adding a special ‘repeat_count’ variable to
the sixel struct, we can initialize it to ‘1’ when we see DECGRI, and
then simply overwrite it as the parameter value gets updated. This
allows us to drop an if..else when emitting the sixel.
When emitting a sixel, we need to:
a) scroll terminal content to ensure the new image fits
b) position the text cursor
Recent changes in the cursor positioning logic meant we reduced the
number of linefeeds, to ensure 1) sixels could be printed to the
bottom row without scrolling the terminal contents, and 2) the cursor
was positioned on the last sixel row.
Except, we’re not actually positioning the cursor on the last sixel
row. We’re positioning it on the text row that maps to the *upper*
pixel of the last sixel.
In most cases, this _is_ the last row of the sixel. But for certain
combinations of font and image sizes, it may be higher up.
This patch fixes a regression, where the terminal contents weren’t
scrolled up enough for certain images, causing a crash when trying to
dirty a not-yet scrolled in row.
The fix is this:
* Always scroll by the number of rows occupied by the image, minus
one. This ensures the image "fits".
* Adjust the cursor position, if necessary.
* Lazy initialize image height. This is necessary to prevent garbage
from being rendered for "empty" sixels.
* Fix plotting of non-1:1 pixels
* Fix calculation of height in resize(), for non-1:1 aspect ratios
That is, parse P1 when initializing a new sixel, and don’t ignore
pad/pad in the raster attributes command.
The default aspect ratio is 2:1, but most sixels will override it in
the raster attributes command (to 1:1).
This adjusts the logic that positions the text cursor after emitting a
sixel, when sixel scrolling mode is *enabled*.
We’ve always mimicked XTerm’s behavior. However, XTerm recently
changed its behavior, to better match that of an VT382.
Now, the cursor is placed *on* the last row of the sixel, instead of
on a new row after the sixel.
This allows applications to print sixels to the bottom row of the
terminal, without causing the content to scroll.
Finally, there was a bug in the horizontal positioning of the cursor;
it was placed on the *first* column of the row, instead of on the
first column of the sixel.
When we try to resize a sixel past the current max height, we set
col > image-width to signal this.
This means ‘width’ could be smaller than ‘col’. When calculating how
many sixels to emit in sixel_add_many(), we didnt’ account for this.
The resulting value was -1, converted to ‘unsigned’. I.e. a very large
value. This resulted in an assert triggering in sixel_add() in debug
builds, and a crash in release builds.
This function reflows all sixels in the specified grid.
The pre-existing sixel_reflow() function is a shortcut for
sixel_reflow_grid(term, &term->normal)
sixel_reflow_grid(term, &term->alt);
Each cell now tracks it’s current color source:
* default fg/bg
* base16 fg/bg (maps to *both* the regular and bright colors)
* base256 fg/bg
* RGB
Note that we don’t have enough bits to separate the regular from the
bright colors. These _shouldn’t_ be the same, so we ought to be
fine...
Instead, do the palette lookup when we receive the DECGCI (i.e. when
the palette entry is selected), and store the actual color value in
our sixel struct.
We have all information we need to calculate the default background
color in sixel_init():
* Whether the image have transparency or not
* The current ANSI background color
If an image was split up across the scrollback, and the first image
chunk was resized due to it being blended with an already existing
sixel, we crashed.
The reason being the second chunk, emitted *after* the scrollback,
tried to memcpy() image data from a now-free:d image buffer.
The fix is pretty simple: create copies for *all* chunks when an image
has to be split up across the scrollback. Previously, the first chunk
re-used the source image buffer.
Closes#608
Writing a sixel on top of an already existing sixel currently has the
following limitations in foot:
* The parts of the first sixel that is covered by the new sixel are
removed, completely. Even if the new sixel has transparent
areas. I.e. writing a transparent sixel on top of another
sixel *replaces* the first sixel with the new sixel, instead of
layering them on top of each other.
* The second sixel erases the first sixel cell-wise. That is, a sixel
whose size isn’t a multiple of the cell dimensions will leave
unsightly holes in the first sixel.
This patch takes care of both issues.
The first one is actually the easiest one: all we need to do is
calculate the intersection, and blend the two images. To keep things
relatively simple, we use the pixman image from the *new* image, and
use the ‘OVER_REVERSE’ operation to blend the new image over the old
one.
That is, the old image is still split into four tiles (top, left,
right, bottom), just like before. But instead of throwing away the
fifth middle tile, we blend it with the new image. As an optimization,
this is only done if the new image has transparency (P1=1).
The second problem is solved by detecting when we’re erasing an area
from the second image that is larger than the new image. In this case,
we enlarge the new image, and copy the old image into the new one.
Finally, when we enlarge the new image, there may be areas in the new
image that is *not* covered by the old image. These areas are made
transparent.
The end result is:
* Each cell is covered by at *most* 1 sixel image. I.e. the total
numbers of sixels are finite. This is important for the ‘mpv
--vo=sixel’ use case - we don’t want to end up with thousands of
sixels layered on top of each other.
* Writing an opaque sixel on top of another sixel has _almost_ zero
performance impact. Especially if the two sixels have the same size,
so that we don’t have to resize the new image. Again, important for
the ‘mpv --vo=sixel’ use case.
Closes#562
If the image was accompanied with a “Set Raster Attributes” (SRA)
command, make sure we *never* shrink the image below the size
specified in the SRA.
Images are normally shrunk when their bottom rows are fully
transparent. This enables sixels that aren’t a multiple of 6 to be
emitted, without also emitting an SRA command.
But if there *is* an SRA command, obey it.
Verified against XTerm-367
sixel_color_set() is called when the number of (sixel) color registers
is changed.
It frees the current palette, and changes the “palette size” variable.
Originally, we only had a single palette. This is the one free:d by
sixel_color_set().
Later, we added support for private vs. shared palettes. With this
change, we now have one palette that is “never” free:d (the shared
one), and a private palette that is always free:d after a sixel has
been emitted.
‘sixel.palette’ is a pointer to the palette currently in use, and
should only be accessed **while emitting a sixel**.
This is the pointer sixel_color_set() free:d. So for example, if
‘sixel.palette’ pointed to the shared palette, we’d free the shared
palette. But, we didn’t reset ‘sixel.shared_palette’, causing a double
free later on.
Closes#427
0 is a perfectly valid row number, and if max_non_empty_row_no==0,
that means we have *1* sixel row, and after trimming the image, the
image will have a height of 6 pixels.
If the sixel sequence is empty (or at least doesn’t emit any non-empty
pixels), then trimming the image should result in an image height of
0.
When max_non_empty_row_no is initialized to -1, it will still have
that value in unhook(), which makes the final image height 0.
term_print() is called whenever the client application “prints”
something to the grid. It is called for both ASCII and UTF-8
characters, and needs to handle sixels, insert mode and ASCII
vs. graphical charsets.
Since it’s on the hot path, this becomes unnecessarily slow.
This patch adds a “fast” version of term_print(), tailored for the
common case: ASCII characters in non-insert mode, without any sixels
and non-graphical charsets.
A new function, term_update_ascii_printer(), has been added, and must
be called whenever:
* The currently selected charset *index* changes
* The currently selected charset changes (from ASCII to graphical, or
vice verse)
* Sixels are added to the grid
* Sixels are removed from the grid
* Insert mode is enabled/disabled
This avoids a call to sixel_overwrite_by_row() (where we also exit
early if the image list is empty).
This saves a couple of instructions to set up the arguments for
sixel_overwrite_by_row().
“current geometry” will report whatever value is the smallest; the max
geometry or the current window size.
But “max geometry” always returns the configured max geometry.
This aligns foot’s behavior with XTerm.
When P2=1, empty pixels are transparent.
This patch also changes the behavior of P2=0|2, from setting empty
pixels to the default background color, to instead use the *current*
background color.
To implement this, a couple of changes are needed:
* Sixel pixels always use alpha=1.0, except for *empty* cells when
P2=1 (i.e. transparent pixels).
* The renderer draws sixels with the OVER operator, instead of the SRC
operator.
* The renderer *must* now render the cells beneath the sixel. As an
optimization, this is only done for sixels where P2=1. I.e. for
fully opaque sixels, there’s no need to render the cells beneath.
The sixel renderer isn’t yet hooked into the multi-threaded
renderer. This means *rows* (not just the cells) beneath
maybe-transparent sixels are rendered single-threaded.
Closes#391.
This ensures we don’t trim off bottom rows in unhook().
This could happen either because the application used “Set Raster
Attributes” to configure an image size larger than the sixels later
emitted.
Or, the last sixel row contains empty pixel rows.
In either case, the size set with “Set Raster Attributes” defines
the *minimum* image size; the image may still end up being larger.
DECGRI, i.e. repeat sixel character, only need to do image resizing,
and updating the current ‘column’ value *once*.
By adding sixel_add_many(), and doing the size/resize checking there,
the performance of sixel_add() is made much simpler.
This boosts performance quite noticeably when the application is
emitting many and/or long repeat sequences.
This results in the same number of instructions inside the loop, with
a ‘lea’ instead of a ‘mov’, but simplifies the post-loop logic to
update the global state.
All-empty pixels rows in the last sixel row should not be included in
the final sixel image.
This allows applications to emit sixels whose height is not a multiple
of 6, and is how XTerm works.
This is done by tracking the largest row number that contains
non-empty pixels.
In unhook, when emitting the image, the image height is adjusted based
on this value.
This is to optimize sixel_add(). It already checks ‘pos’, to see if it
needs to resize the image, and if the resize fails (either due to
allocation failures, or because we’ve reached the maximum width), we
bail out.
We need to do the same thing for height. But, we don’t want to add
another check to sixel_add() since its’s performance critical.
By setting ‘col’ outside the image boundaries when ‘row’ reaches the
maximum image height, we can “re-use” the existing code in
sixel_add().
