The Tor Browser: gfx/cairo/libpixman/src/refactor@6474c204b198 (annotated)

gfx/cairo/libpixman/src/refactor@6474c204b198 (annotated)

gfx/cairo/libpixman/src/refactor

Wed, 31 Dec 2014 06:09:35 +0100

author: Michael Schloh von Bennewitz <michael@schloh.com>
date: Wed, 31 Dec 2014 06:09:35 +0100
changeset 0: 6474c204b198
permissions: -rw-r--r--

Cloned upstream origin tor-browser at tor-browser-31.3.0esr-4.5-1-build1
revision ID fc1c9ff7c1b2defdbc039f12214767608f46423f for hacking purpose.

 Roadmap
 - Move all the fetchers etc. into pixman-image to make pixman-compose.c
   less intimidating.
   DONE
 - Make combiners for unified alpha take a mask argument. That way
   we won't need two separate paths for unified vs component in the
   general compositing code.
   DONE, except that the Altivec code needs to be updated. Luca is
   looking into that.
 - Delete separate 'unified alpha' path
   DONE
 - Split images into their own files
   DONE
 - Split the gradient walker code out into its own file
   DONE
 - Add scanline getters per image
   DONE
 - Generic 64 bit fetcher
   DONE
 - Split fast path tables into their respective architecture dependent
   files.
 See "Render Algorithm" below for rationale
 Images will eventually have these virtual functions:
        get_scanline()
        get_scanline_wide()
        get_pixel()
        get_pixel_wide()
        get_untransformed_pixel()
        get_untransformed_pixel_wide()
        get_unfiltered_pixel()
        get_unfiltered_pixel_wide()
        store_scanline()
        store_scanline_wide()
 .
 Initially we will just have get_scanline() and get_scanline_wide();
 these will be based on the ones in pixman-compose. Hopefully this will
 reduce the complexity in pixman_composite_rect_general().
 Note that there is access considerations - the compose function is
 being compiled twice.
 .
 Split image types into their own source files. Export noop virtual
 reinit() call.  Call this whenever a property of the image changes.
 .
 Split the get_scanline() call into smaller functions that are
 initialized by the reinit() call.
 The Render Algorithm:
 	(first repeat, then filter, then transform, then clip)
 Starting from a destination pixel (x, y), do
 x = x - xDst + xSrc
 	  y = y - yDst + ySrc
 reject pixel that is outside the clip
 	This treats clipping as something that happens after
 	transformation, which I think is correct for client clips. For
 	hierarchy clips it is wrong, but who really cares? Without
 	GraphicsExposes hierarchy clips are basically irrelevant. Yes,
 	you could imagine cases where the pixels of a subwindow of a
 	redirected, transformed window should be treated as
 	transparent. I don't really care
 	Basically, I think the render spec should say that pixels that
 	are unavailable due to the hierarcy have undefined content,
 	and that GraphicsExposes are not generated. Ie., basically
 	that using non-redirected windows as sources is fail. This is
 	at least consistent with the current implementation and we can
 	update the spec later if someone makes it work.
 	The implication for render is that it should stop passing the
 	hierarchy clip to pixman. In pixman, if a souce image has a
 	clip it should be used in computing the composite region and
 	nowhere else, regardless of what "has_client_clip" says. The
 	default should be for there to not be any clip.
 	I would really like to get rid of the client clip as well for
 	source images, but unfortunately there is at least one
 	application in the wild that uses them.
 Transform pixel: (x, y) = T(x, y)
 Call p = GetUntransformedPixel (x, y)
 If the image has an alpha map, then
 		Call GetUntransformedPixel (x, y) on the alpha map
 		add resulting alpha channel to p
 	   return p
 	Where GetUnTransformedPixel is:
 switch (filter)
 	  {
 	  case NEAREST:
 		return GetUnfilteredPixel (x, y);
 		break;
 	  case BILINEAR:
 		return GetUnfilteredPixel (...) // 4 times
 		break;
 	  case CONVOLUTION:
 		return GetUnfilteredPixel (...) // as many times as necessary.
 		break;
 	  }
 	Where GetUnfilteredPixel (x, y) is
 switch (repeat)
 	   {
 	   case REPEAT_NORMAL:
 	   case REPEAT_PAD:
 	   case REPEAT_REFLECT:
 		// adjust x, y as appropriate
 		break;
 	   case REPEAT_NONE:
 	        if (x, y) is outside image bounds
 		     return 0;
 		break;
 	   }
 	   return GetRawPixel(x, y)
 	Where GetRawPixel (x, y) is
 Compute the pixel in question, depending on image type.
 For gradients, repeat has a totally different meaning, so
 UnfilteredPixel() and RawPixel() must be the same function so that
 gradients can do their own repeat algorithm.
 So, the GetRawPixel
 	for bits must deal with repeats
 	for gradients must deal with repeats (differently)
 	for solids, should ignore repeats.
 	for polygons, when we add them, either ignore repeats or do
 	something similar to bits (in which case, we may want an extra
 	layer of indirection to modify the coordinates).
 It is then possible to build things like "get scanline" or "get tile" on
 top of this. In the simplest case, just repeatedly calling GetPixel()
 would work, but specialized get_scanline()s or get_tile()s could be
 plugged in for common cases.
 By not plugging anything in for images with access functions, we only
 have to compile the pixel functions twice, not the scanline functions.
 And we can get rid of fetchers for the bizarre formats that no one
 uses. Such as b2g3r3 etc. r1g2b1? Seriously? It is also worth
 considering a generic format based pixel fetcher for these edge cases.
 Since the actual routines depend on the image attributes, the images
 must be notified when those change and update their function pointers
 appropriately. So there should probably be a virtual function called
 (* reinit) or something like that.
 There will also be wide fetchers for both pixels and lines. The line
 fetcher will just call the wide pixel fetcher. The wide pixel fetcher
 will just call expand, except for 10 bit formats.
 Rendering pipeline:
 Drawable:
 . if (picture has alpha map)
 .1. Position alpha map according to the alpha_x/alpha_y
 .2. Where the two drawables intersect, the alpha channel
 		     Replace the alpha channel of source with the one
 		     from the alpha map. Replacement only takes place
 		     in the intersection of the two drawables' geometries.
 . Repeat the drawable according to the repeat attribute
 . Reconstruct a continuous image according to the filter
 . Transform according to the transform attribute
 . Position image such that src_x, src_y is over dst_x, dst_y
 . Sample once per destination pixel
 . Clip. If a pixel is not within the source clip, then no
 	   compositing takes place at that pixel. (Ie., it's *not*
 	   treated as 0).
 	Sampling a drawable:
 	- If the channel does not have an alpha channel, the pixels in it
 	  are treated as opaque.
 	Note on reconstruction:
 	- The top left pixel has coordinates (0.5, 0.5) and pixels are
 	  spaced 1 apart.
 Gradient:
 . Unless gradient type is conical, repeat the underlying (0, 1)
 		gradient according to the repeat attribute
 . Integrate the gradient across the plane according to type.
 . Transform according to transform attribute
 . Position gradient
 . Sample once per destination pixel.
 . Clip
 Solid Fill:
 . Repeat has no effect
 . Image is already continuous and defined for the entire plane
 . Transform has no effect
 . Positioning has no effect
 . Sample once per destination pixel.
 . Clip
 Polygon:
 . Repeat has no effect
 . Image is already continuous and defined on the whole plane
 . Transform according to transform attribute
 . Position image
 . Supersample 15x17 per destination pixel.
 . Clip
 Possibly interesting additions:
 	- More general transformations, such as warping, or general
 	  shading.
 	- Shader image where a function is called to generate the
           pixel (ie., uploading assembly code).
 	- Resampling kernels
 	  In principle the polygon image uses a 15x17 box filter for
 	  resampling. If we allow general resampling filters, then we
 	  get all the various antialiasing types for free.
 	  Bilinear downsampling looks terrible and could be much
 	  improved by a resampling filter. NEAREST reconstruction
 	  combined with a box resampling filter is what GdkPixbuf
 	  does, I believe.
 	  Useful for high frequency gradients as well.
 	  (Note that the difference between a reconstruction and a
 	  resampling filter is mainly where in the pipeline they
 	  occur. High quality resampling should use a correctly
 	  oriented kernel so it should happen after transformation.
 	  An implementation can transform the resampling kernel and
 	  convolve it with the reconstruction if it so desires, but it
 	  will need to deal with the fact that the resampling kernel
 	  will not necessarily be pixel aligned.
 	  "Output kernels"
 	  One could imagine doing the resampling after compositing,
 	  ie., for each destination pixel sample each source image 16
 	  times, then composite those subpixels individually, then
 	  finally apply a kernel.
 	  However, this is effectively the same as full screen
 	  antialiasing, which is a simpler way to think about it. So
 	  resampling kernels may make sense for individual images, but
 	  not as a post-compositing step.
 	  Fullscreen AA is inefficient without chained compositing
 	  though. Consider an (image scaled up to oversample size IN
 	  some polygon) scaled down to screen size. With the current
 	  implementation, there will be a huge temporary. With chained
 	  compositing, the whole thing ends up being equivalent to the
 	  output kernel from above.
 	- Color space conversion
 	  The complete model here is that each surface has a color
 	  space associated with it and that the compositing operation
 	  also has one associated with it. Note also that gradients
 	  should have associcated colorspaces.
 	- Dithering
 	  If people dither something that is already dithered, it will
 	  look terrible, but don't do that, then. (Dithering happens
 	  after resampling if at all - what is the relationship
 	  with color spaces? Presumably dithering should happen in linear
 	  intensity space).
 	- Floating point surfaces, 16, 32 and possibly 64 bit per
 	  channel.
 	Maybe crack:
 	- Glyph polygons
 	  If glyphs could be given as polygons, they could be
 	  positioned and rasterized more accurately. The glyph
 	  structure would need subpixel positioning though.
 	- Luminance vs. coverage for the alpha channel
 	  Whether the alpha channel should be interpreted as luminance
           modulation or as coverage (intensity modulation). This is a
           bit of a departure from the rendering model though. It could
 	  also be considered whether it should be possible to have
 	  both channels in the same drawable.
 	- Alternative for component alpha
 	  - Set component-alpha on the output image.
 	    - This means each of the components are sampled
 	      independently and composited in the corresponding
 	      channel only.
 	  - Have 3 x oversampled mask
 	  - Scale it down by 3 horizontally, with [ 1/3, 1/3, 1/3 ]
             resampling filter.
 	    Is this equivalent to just using a component alpha mask?
 	Incompatible changes:
 	- Gradients could be specified with premultiplied colors. (You
 	  can use a mask to get things like gradients from solid red to
 	  transparent red.
 Refactoring pixman
 The pixman code is not particularly nice to put it mildly. Among the
 issues are
 - inconsistent naming style (fb vs Fb, camelCase vs
   underscore_naming). Sometimes there is even inconsistency *within*
   one name.
       fetchProc32 ACCESS(pixman_fetchProcForPicture32)
   may be one of the uglies names ever created.
   coding style:
   	 use the one from cairo except that pixman uses this brace style:
 		while (blah)
 		{
 		}
 	Format do while like this:
 	       do
 	       {
 	       }
 	       while (...);
 - PIXMAN_COMPOSITE_RECT_GENERAL() is horribly complex
 - switch case logic in pixman-access.c
   Instead it would be better to just store function pointers in the
   image objects themselves,
   	get_pixel()
 	get_scanline()
 - Much of the scanline fetching code is for formats that no one
   ever uses. a2r2g2b2 anyone?
   It would probably be worthwhile having a generic fetcher for any
   pixman format whatsoever.
 - Code related to particular image types should be split into individual
   files.
 	pixman-bits-image.c
 	pixman-linear-gradient-image.c
 	pixman-radial-gradient-image.c
 	pixman-solid-image.c
 - Fast path code should be split into files based on architecture:
        pixman-mmx-fastpath.c
        pixman-sse2-fastpath.c
        pixman-c-fastpath.c
        etc.
   Each of these files should then export a fastpath table, which would
   be declared in pixman-private.h. This should allow us to get rid
   of the pixman-mmx.h files.
   The fast path table should describe each fast path. Ie there should
   be bitfields indicating what things the fast path can handle, rather than
   like now where it is only allowed to take one format per src/mask/dest. Ie.,
   {
     FAST_a8r8g8b8 | FAST_x8r8g8b8,
     FAST_null,
     FAST_x8r8g8b8,
     FAST_repeat_normal | FAST_repeat_none,
     the_fast_path
   }
 There should then be *one* file that implements pixman_image_composite().
 This should do this:
      optimize_operator();
      convert 1x1 repeat to solid (actually this should be done at
      image creation time).
      is there a useful fastpath?
 There should be a file called pixman-cpu.c that contains all the
 architecture specific stuff to detect what CPU features we have.
 Issues that must be kept in mind:
        - we need accessor code to be preserved
        - maybe there should be a "store_scanline" too?
          Is this sufficient?
 	 We should preserve the optimization where the
 	 compositing happens directly in the destination
 	 whenever possible.
 	- It should be possible to create GPU samplers from the
 	  images.
 The "horizontal" classification should be a bit in the image, the
 "vertical" classification should just happen inside the gradient
 file. Note though that
       (a) these will change if the tranformation/repeat changes.
       (b) at the moment the optimization for linear gradients
           takes the source rectangle into account. Presumably
 	  this is to also optimize the case where the gradient
 	  is close enough to horizontal?
 Who is responsible for repeats? In principle it should be the scanline
 fetch. Right now NORMAL repeats are handled by walk_composite_region()
 while other repeats are handled by the scanline code.
 (Random note on filtering: do you filter before or after
 transformation?  Hardware is going to filter after transformation;
 this is also what pixman does currently). It's not completely clear
 what filtering *after* transformation means. One thing that might look
 good would be to do *supersampling*, ie., compute multiple subpixels
 per destination pixel, then average them together.

The Tor Browser / annotate

gfx/cairo/libpixman/src/refactor@6474c204b198 (annotated)

gfx/cairo/libpixman/src/refactor