The wording in the original VT340 documentation is vague:
Pixel positions specified as 0 are set to the current background
color.
What does that mean? We _thought_ it meant the current ANSI background
color, as set with e.g. CSI 4x m.
It's still all a bit vague, but seeing that we have separate palettes
for text and graphic (should we?), it doesn't make sense to use the
ANSI background color as the default sixel background color.
So, use entry 0 from the sixel palette instead.
The things affecting which ASCII printer we use have grown...
Instead of checking everything inside term_update_ascii_printer(), use
a bitfield.
Anything affecting the printer used, must now set a bit in this
bitfield. This makes term_update_ascii_printer() much faster, since
all it needs to do is check if the bitfield is zero or not.
Changing pan/pad changes the sixel's aspect ratio. While I don't know
for certain what a real VT340 would do, I suspect it would change the
aspect ratio of all subsequent sixels, but not those already emitted.
The way we implement sixels in foot, makes this behavior hard to
implement. We currently don't resize the image properly if the aspect
ratio is changed, but not the RA area. We have code that assumes all
sixel lines have the same aspect ratio, etc.
Since no "normal" applications change the aspect ratio in the middle
of a sixel, simply disallow it, and print a warning.
This also fixes a crash, when writing sixels after having modified the
aspect ratio.
In some cases, a sixel may be resized vertically, while still having a
zero-width. In this case, the resize operations are short-cutted, and
no actual allocations are done.
However, we forgot to set 'alloc_height' when doing so. As a result,
the trimming code (when the sixel is "done"), trimmed away the entire
sixel.
wmemset() is heavily optimized, and in some cases, *much* faster than
manually initializing the new image pixels.
Furthermore, assume calloc() is better at initializing memory to zero,
and use that when initializing new pixels in a transparent image.
When trimming trailing empty rows from the final image, handle the RA
region correctly:
* If image is transparent, trim down to the last non-empty pixel row,
regardless of whether it is inside or outside the RA region.
* If image is opaque, trim down to either the last non-empty pixel
row, or the RA region, whatever is the largest height.
The raster attributes is, really, just a way to erase an area. And, if
the sixel is transparent, it's a nop.
The final text cursor position depends, not on our image size (which
is based on RA), but on the final graphical cursor position.
However, we do want to continue using it as a hint of the final image
size, to be able to pre-allocate the backing buffer.
So, here's what we do:
* When trimming trailing transparent rows, only trim the *image*, if
the graphical cursor is positioned on the last sixel row, *and* the
sixel is transparent.
* Opaque sixels aren't trimmed at all, since RA in this acts as an
erase that fills the RA region with the background color.
* The graphical cursor position is always adjusted (i.e. trimmed),
since it affects the text cursor position.
* The text cursor position is now calculated from the graphical cursor
position, instead of the image height.
After emitting a sixel, place the cursor on the character row touched
by the last sixel. The last sixel _may_ not be a multiple of 6
pixels, *if* the sixel had an explicit width/height set via raster
attributes.
This is an intended deviation from the DEC cursor placement algorithm,
where the cursor is placed on the character row touched by the last
sixel's *upper* pixel.
The adjusted algorithm implemented here makes it much easier for
applications to handle text-after-sixels, since they can now simply
assume a single text newline is required to move the cursor to a
character row not touched by the sixel.
* Store pointer to current pixel (i.e. pixel we're about to write to),
instead of a row-byte-offset. This way, we don't have to calculate the
offset into the backing image every time we emit a sixel band.
* Pass data pointer directly to sixel_add_*(), to avoid having to
calculate an offset into the backing image.
* Special case adding a single 1:1 sixel. This removes a for loop, and
simplifies state (position) updates. It is likely LTO does this for
us, but this way, we get it optimized in non-LTO builds as well.
In resize_vertically(), we assumed a sixel is 6 pixels tall. This is
mostly true, but not for non-1:1 sixels. Or, to be more precise, not
for sixels where 'pan' != 1.
This caused us to allocate too little backing memory, resulting in a
crash when we later tried to write to the image.
When erasing a sixel, the cells underneath it must be marked as
'dirty', in order to be re-rendered.
This was not being done correctly; the for loop loops *from* the start
col, meaning the *end* col is *not* sixel->pos.col, as that's
the *number* of columns, not the *end* column.
Before this patch, when the cell dimensions changed (i.e. when the
font size changes), sixel images were either removed (the new cell
dimensions are smaller than the old), or simply kept at their original
size (new cell dimensions are larger).
With this patch, sixels are instead resized. This means a
sixel *always* occupies the same number of rows and columns,
regardless of how much the font size is changed.
This is done by maintaining two sets of image data and pixman images,
as well as their dimensions. These two sets are the new ‘original’ and
‘scaled’ members of the sixel struct.
The "top-level" pixman image pointer, and the ‘width’ and ‘height’
members either point to the "original", or the "scaled" version.
They are invalidated as soon as the cell dimensions change. They, and
the ‘scaled’ image is updated on-demand (when we need to render a
sixel).
Note that the ‘scaled’ image is always NULL when the current cell
dimensions matches the ones used when emitting the sixel (to save
run-time memory).
Closes#1383
Images with an aspect ratio of 1:1 are by far the most common (though
not the default).
It makes a lot of sense, performance wise, to special case
them.
Specifically, the sixel_add() function benefits greatly from this, as
it is the inner most, most heavily executed function when parsing a
sixel image.
sixel_add_many() also benefits, since allows us to drop a
multiplication. Since sixel_add_many() always called first (no other
call sites call sixel_add() directly), this has a noticeable effect on
performance.
Another thing that helps (though not as much), and not specifically
with AR 1:1 images, is special casing DECGRI a bit.
Up until now, it simply updated the current sixel parameter value. The
problem is that the default parameter value is 0. But, a value of 0
should be treated as 1. By adding a special ‘repeat_count’ variable to
the sixel struct, we can initialize it to ‘1’ when we see DECGRI, and
then simply overwrite it as the parameter value gets updated. This
allows us to drop an if..else when emitting the sixel.
When emitting a sixel, we need to:
a) scroll terminal content to ensure the new image fits
b) position the text cursor
Recent changes in the cursor positioning logic meant we reduced the
number of linefeeds, to ensure 1) sixels could be printed to the
bottom row without scrolling the terminal contents, and 2) the cursor
was positioned on the last sixel row.
Except, we’re not actually positioning the cursor on the last sixel
row. We’re positioning it on the text row that maps to the *upper*
pixel of the last sixel.
In most cases, this _is_ the last row of the sixel. But for certain
combinations of font and image sizes, it may be higher up.
This patch fixes a regression, where the terminal contents weren’t
scrolled up enough for certain images, causing a crash when trying to
dirty a not-yet scrolled in row.
The fix is this:
* Always scroll by the number of rows occupied by the image, minus
one. This ensures the image "fits".
* Adjust the cursor position, if necessary.
* Lazy initialize image height. This is necessary to prevent garbage
from being rendered for "empty" sixels.
* Fix plotting of non-1:1 pixels
* Fix calculation of height in resize(), for non-1:1 aspect ratios
That is, parse P1 when initializing a new sixel, and don’t ignore
pad/pad in the raster attributes command.
The default aspect ratio is 2:1, but most sixels will override it in
the raster attributes command (to 1:1).
This adjusts the logic that positions the text cursor after emitting a
sixel, when sixel scrolling mode is *enabled*.
We’ve always mimicked XTerm’s behavior. However, XTerm recently
changed its behavior, to better match that of an VT382.
Now, the cursor is placed *on* the last row of the sixel, instead of
on a new row after the sixel.
This allows applications to print sixels to the bottom row of the
terminal, without causing the content to scroll.
Finally, there was a bug in the horizontal positioning of the cursor;
it was placed on the *first* column of the row, instead of on the
first column of the sixel.
When we try to resize a sixel past the current max height, we set
col > image-width to signal this.
This means ‘width’ could be smaller than ‘col’. When calculating how
many sixels to emit in sixel_add_many(), we didnt’ account for this.
The resulting value was -1, converted to ‘unsigned’. I.e. a very large
value. This resulted in an assert triggering in sixel_add() in debug
builds, and a crash in release builds.
This function reflows all sixels in the specified grid.
The pre-existing sixel_reflow() function is a shortcut for
sixel_reflow_grid(term, &term->normal)
sixel_reflow_grid(term, &term->alt);
Each cell now tracks it’s current color source:
* default fg/bg
* base16 fg/bg (maps to *both* the regular and bright colors)
* base256 fg/bg
* RGB
Note that we don’t have enough bits to separate the regular from the
bright colors. These _shouldn’t_ be the same, so we ought to be
fine...
Instead, do the palette lookup when we receive the DECGCI (i.e. when
the palette entry is selected), and store the actual color value in
our sixel struct.
We have all information we need to calculate the default background
color in sixel_init():
* Whether the image have transparency or not
* The current ANSI background color
If an image was split up across the scrollback, and the first image
chunk was resized due to it being blended with an already existing
sixel, we crashed.
The reason being the second chunk, emitted *after* the scrollback,
tried to memcpy() image data from a now-free:d image buffer.
The fix is pretty simple: create copies for *all* chunks when an image
has to be split up across the scrollback. Previously, the first chunk
re-used the source image buffer.
Closes#608
Writing a sixel on top of an already existing sixel currently has the
following limitations in foot:
* The parts of the first sixel that is covered by the new sixel are
removed, completely. Even if the new sixel has transparent
areas. I.e. writing a transparent sixel on top of another
sixel *replaces* the first sixel with the new sixel, instead of
layering them on top of each other.
* The second sixel erases the first sixel cell-wise. That is, a sixel
whose size isn’t a multiple of the cell dimensions will leave
unsightly holes in the first sixel.
This patch takes care of both issues.
The first one is actually the easiest one: all we need to do is
calculate the intersection, and blend the two images. To keep things
relatively simple, we use the pixman image from the *new* image, and
use the ‘OVER_REVERSE’ operation to blend the new image over the old
one.
That is, the old image is still split into four tiles (top, left,
right, bottom), just like before. But instead of throwing away the
fifth middle tile, we blend it with the new image. As an optimization,
this is only done if the new image has transparency (P1=1).
The second problem is solved by detecting when we’re erasing an area
from the second image that is larger than the new image. In this case,
we enlarge the new image, and copy the old image into the new one.
Finally, when we enlarge the new image, there may be areas in the new
image that is *not* covered by the old image. These areas are made
transparent.
The end result is:
* Each cell is covered by at *most* 1 sixel image. I.e. the total
numbers of sixels are finite. This is important for the ‘mpv
--vo=sixel’ use case - we don’t want to end up with thousands of
sixels layered on top of each other.
* Writing an opaque sixel on top of another sixel has _almost_ zero
performance impact. Especially if the two sixels have the same size,
so that we don’t have to resize the new image. Again, important for
the ‘mpv --vo=sixel’ use case.
Closes#562
