Add VAVDA, the VAAPI Video Decode Accelerator for Intel CPUs.

content/common/gpu/media/vaapi_h264_decoder.h

+// Copyright (c) 2012 The Chromium Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+//
+// This file contains an implementation of a class that provides H264 decode
+// support for use with VAAPI hardware video decode acceleration on Intel
+// systems.
+
+#ifndef CONTENT_COMMON_GPU_MEDIA_VAAPI_H264_DECODER_H_
+#define CONTENT_COMMON_GPU_MEDIA_VAAPI_H264_DECODER_H_
+
+#include <vector>
+#include <queue>
+
+#include <GL/glx.h>
+
+#include "base/callback_forward.h"
+#include "base/lazy_instance.h"
+#include "base/memory/linked_ptr.h"
+#include "base/memory/scoped_ptr.h"
+#include "content/common/gpu/media/h264_dpb.h"
+#include "content/common/gpu/media/h264_parser.h"
+#include "media/base/video_decoder_config.h"
+#include "third_party/libva/va/va.h"
+
+namespace content {

[Ami GONE FROM CHROMIUM, 2012/04/09 02:41:47]

This means you can drop content:: from everywhere below.
(please make a pass over the entire CL for this)

[Pawel Osciak, 2012/05/03 16:22:07]

Done.

+
+// An H264 decoder for use for VA-API-specific decoding. Provides features not
+// supported by libva, including stream parsing, reference picture management
+// and other operations not supported by the HW codec.
+//
+// Provides functionality to allow plugging VAAPI HW acceleration into the
+// VDA framework.
+//
+// Clients of this class are expected to pass H264 Annex-B byte stream and
+// will receive decoded pictures via client-provided |PictureReadyCallback|.
+//
+// If used in multi-threaded environment, some of the functions have to be
+// called on the display thread (i.e. the thread that has the GLX context

[Ami GONE FROM CHROMIUM, 2012/04/09 02:41:47]

Per my comments on the previous PS, I'd refer to the ChildThread (a.k.a. main
thread of the GPU process) and the Decoder thread (a.k.a. the thread started by
VAVDA).

[Pawel Osciak, 2012/05/03 16:22:07]

Can't emphasize enough how awkward that is for me, but I give up :)

+// passed to Initialize() set as current). This is essential so that the GLX
+// calls can work properly. That thread is further referred to as "GLX thread".
+// Decoder thread does not require GLX context and should be the same
+// as the one on which Decode*() functions are called.
+class VaapiH264Decoder {
+ public:
+  // Callback invoked on the client when a picture is to be displayed.
+  // Callee has to call PutPicToTexture() for the given picture before
+  // displaying it, to ensure the contents have been synced properly.
+  // Arguments: input buffer id, output buffer id (both provided by the client
+  // at the time of Decode() and AssignPictureBuffer() calls).
+  typedef base::Callback<void(int32, int32)> OutputPicCallback;

[Ami GONE FROM CHROMIUM, 2012/04/09 02:41:47]

nit: FWIW new-style callbacks (which this is) are referred to using FooCB and
foo_cb_ in media code.  
(b/c the old-style callbacks used FooCallback and foo_callback_ and it was handy
during the migration to have a different convention for the new type to tell
what was what, and now that the migration is over we're enjoying the shorter
type & variable names).

[Pawel Osciak, 2012/05/03 16:22:07]

Done.

+
+  // Decode result codes.
+  enum DecResult {

[Ami GONE FROM CHROMIUM, 2012/04/09 02:41:47]

s/DecResult/DecodeResult/
but isn't this really DecoderState?

[Pawel Osciak, 2012/05/03 16:22:07]

No, it's the result. The enum for State is further below.

+    kDecodeError,  // Error while decoding.
+    // TODO posciak: unsupported streams are currently treated as error
+    // in decoding; in future it could perhaps be possible to fall back
+    // to software decoding instead.
+    // kStreamError,  // Error in stream.
+    kReadyToDecode,  // Successfully initialized.
+    kDecodedFrame,  // Successfully decoded a frame.
+    kNeedMoreStreamData,  // Need more stream data to decode the next frame.
+    kNoOutputAvailable,  // Waiting for the client to free up output surfaces.
+  };
+
+  VaapiH264Decoder();
+  // Should be called on the GLX thread, for the surface cleanup to work
+  // properly.
+  ~VaapiH264Decoder();
+
+  // Initializes and sets up libva connection and GL/X11 resources.
+  // Must be called on the GLX thread with |glx_context| being current and
+  // with decoder thread not yet running.
+  // |output_pic_callback| will be called to notify when a picture can be
+  // displayed.
+  bool Initialize(media::VideoCodecProfile profile,
+                  Display* x_display,
+                  GLXContext glx_context,
+                  const OutputPicCallback& output_pic_callback);
+  void Destroy();
+
+  // Notify the decoder that this output buffer has been consumed and
+  // can be reused (overwritten).
+  // Must be run on the decoder thread.
+  void ReusePictureBuffer(int32 picture_buffer_id);
+
+  // Give a new picture buffer (texture) to decoder for use.
+  // Must be run on the GLX thread with decoder thread not yet running.
+  bool AssignPictureBuffer(int32 picture_buffer_id, uint32 texture_id);
+
+  // Sync the data so that the texture for given |picture_buffer_id| can
+  // be displayed.
+  // Must be run on the GLX thread.
+  bool PutPicToTexture(int32 picture_buffer_id);
+
+  // Have the decoder flush its state and trigger output of all previously
+  // decoded pictures via OutputPicCallback.
+  // Returns false if any of the resulting invocations of the callback fail.

[Ami GONE FROM CHROMIUM, 2012/04/09 02:41:47]

I suspect this is in the realm of DCHECK (i.e. programming errors or platform
errors).  Does this really need to support returning a bool?
As support for my skepticism I present the evidence that the (only) caller of
this function does not inspect its return value.

Generally coding an API whose callers must inspect a return value to look for
errors is bug-prone, if that return value is not otherwise inspected.  IOW if
you're returning a Something* then it's easy to use NULL as an error condition,
and callers will be looking at that ret value anyway b/c the "happy" codepath
must use it, too.  But if a function is otherwise void then it's really easy for
callers to forget to inspect the return value.  In these cases it's much more
useful if the API instead has an out-of-band error-reporting mechanism, such as
an OnError callback where all error handling can be centralized.

[Pawel Osciak, 2012/05/03 16:22:07]

There are two callers:
1. VAVDA on Flush request from up the stack, which indeed doesn't check for the
return value because the VDA interface does not care (VDA::Flush is void).
2. Flush in VaapiH264Decoder::StartNewFrame(), which checks and handles it.
Right now I have an API that returns DecResult with a detailed error. The
internal functions return bool. I feel uncomfortable with replacing internal
error checks with an internal callback, is that what you are suggesting? I would
much prefer adding WARN_UNUSED_RESULT.

[Ami GONE FROM CHROMIUM, 2012/05/03 23:22:53]

Of course the VDA interface *does* care; the fact that VDA::Flush() is void is a
consequence of it being an async interface.  The return value of
decoder->Flush() needs to be inspected and NotifyError() in case of failure. 
Added comment there.
I missed this one, somehow.
Ok.  W_U_R is probably worth doing.  Sadly, it isn't as awesome on most (all?)
platforms as you'd like it to be (there are lots of violations it can't detect).

[Pawel Osciak, 2012/05/06 17:49:19]

Right. Although in this case, ignoring it does not break things, so I was
leaving the choice up to the client.

+  bool Flush();
+
+  // Called while decoding.
+  // Stop decoding, discarding all remaining input/output, but do not flush
+  // state, so the playback of the same stream can be resumed (possibly from
+  // another location).
+  void Reset();
+
+  // Set current stream data pointer to |ptr| and |size|.
+  // Must be run on decoder thread.
+  void SetStream(uint8* ptr, size_t size);
+
+  // Start parsing stream to detect picture sizes. Does not produce any
+  // decoded pictures and can be called without providing output textures.
+  // Also to be used after Reset() to find a suitable location in the
+  // stream to resume playback from.
+  DecResult DecodeInitial(int32 input_id);
+
+  // Runs until a frame is decoded or end of provided stream data buffer
+  // is reached. Decoded pictures will be returned asynchronously via
+  // OutputPicCallback.
+  DecResult DecodeOneFrame(int32 input_id);
+
+  // Return dimensions for output buffer (texture) allocation.
+  // Valid only after a successful DecodeInitial().
+  int pic_height() { return pic_height_; }
+  int pic_width() { return pic_width_; }
+
+  // Return the number of output pictures required for decoding.
+  // Valid after a successful DecodeInitial().
+  static size_t GetRequiredNumOfPictures();

[Ami GONE FROM CHROMIUM, 2012/04/09 02:41:47]

Why size_t (an unsigned type)?
In practice we're talking about values in the range of 2-10.  Introducing the
complexity of unsigned arithmetic for this is unwise.

[Pawel Osciak, 2012/05/03 16:22:07]

Mostly because I'm comparing the return value against size() of a vector in one
case...

+
+ private:
+  // We need to keep at least kDPBMaxSize pictures in DPB for
+  // reference/to display later and an additional one for the one currently
+  // being decoded. We also ask for some additional ones since VDA needs
+  // to accumulate a few ready-to-output pictures before it actually starts
+  // displaying and giving them back.
+  enum { kNumReqPictures = H264DPB::kDPBMaxSize + 6 };

[Ami GONE FROM CHROMIUM, 2012/04/09 02:41:47]

Please replace the "6" with the explicit math.
I believe you want 
kDPBMaxSize + media::limits::kMaxVideoFrames
(previously I had a +1 for your "an additional one for the one currently being
decoded" but now I think that's part of the m::l:kMVF number, no?)

[Pawel Osciak, 2012/05/03 16:22:07]

It'll have to be kMVF + 1, as one is also "in flight" in the decoder.

+
+  // Internal state of the decoder.
+  enum State {
+    kUninitialized,  // Initialize() not yet called.
+    kInitialized,  // Initialize() called, pictures requested.
+    kDecoding,  // DecodeInitial() successful, output surfaces allocated.
+    kAfterReset,  // After Reset() during decoding.
+    kError,  // Error in kDecoding state.
+  };
+
+  // Get usable framebuffer configuration for use in binding textures
+  // or return false on failure.
+  bool InitializeFBConfig();
+
+  // Process H264 stream structures.
+  bool ProcessSPS(int sps_id);
+  bool ProcessPPS(int pps_id);
+  bool ProcessSlice(content::H264SliceHeader* slice_hdr);
+
+  // Initialize the current picture according to data in |slice_hdr|.
+  bool InitCurrPicture(content::H264SliceHeader* slice_hdr);
+
+  // Calculate picture order counts for the new picture
+  // on initialization of a new frame (see spec).
+  bool CalculatePicOrderCounts(content::H264SliceHeader* slice_hdr);
+
+  // Update PicNum values in pictures stored in DPB on creation of new
+  // frame (see spec).
+  void UpdatePicNums();
+
+  // Construct initial reference picture lists for use in decoding of
+  // P and B pictures (see 8.2.4 in spec).
+  void ConstructReferencePicListsP(content::H264SliceHeader* slice_hdr);
+  void ConstructReferencePicListsB(content::H264SliceHeader* slice_hdr);
+
+  // Helper functions for reference list construction, per spec.
+  int PicNumF(H264Picture *pic);
+  int LongTermPicNumF(H264Picture *pic);
+
+  // Perform the reference picture lists' modification (reordering), as
+  // specified in spec (8.2.4).
+  //
+  // |list| indicates list number and should be either 0 or 1.
+  bool ModifyReferencePicList(content::H264SliceHeader *slice_hdr, int list);
+
+  // Perform reference picture memory management operations (marking/unmarking
+  // of reference pictures, long term picture management, discarding, etc.).
+  // See 8.2.5 in spec.
+  bool HandleMemoryManagementOps();
+  void ReferencePictureMarking();
+
+  // Start processing a new frame.
+  bool StartNewFrame(content::H264SliceHeader* slice_hdr);
+
+  // All data for a frame received, process it and decode.
+  bool FinishPrevFrameIfPresent();
+
+  // Called after decoding, performs all operations to be done after decoding,
+  // including DPB management, reference picture marking and memory management
+  // operations.
+  // This will also output a picture if one is ready for output.
+  bool FinishPicture();
+
+  // Convert VideoCodecProfile to VAProfile and set it as active.
+  bool SetProfile(media::VideoCodecProfile profile);
+
+  // Vaapi-related functions.
+
+  // Allocates VASurfaces and creates a VAContext for them.
+  bool CreateVASurfaces();
+
+  // Destroys allocated VASurfaces and related VAContext.
+  void DestroyVASurfaces();
+
+  // These queue up data for HW decoder to be committed on running HW decode.
+  bool SendPPS();
+  bool SendIQMatrix();
+  bool SendVASliceParam(content::H264SliceHeader* slice_hdr);
+  bool SendSliceData(const uint8* ptr, size_t size);
+  bool QueueSlice(content::H264SliceHeader* slice_hdr);
+
+  // Helper methods for filling HW structures.
+  void FillVAPicture(VAPictureH264 *va_pic, H264Picture* pic);
+  int FillVARefFramesFromDPB(VAPictureH264 *va_pics, int num_pics);
+
+  // Commits all pending data for HW decoder and starts HW decoder.
+  bool DecodePicture();
+
+  // Notifies client that a picture is ready for output.
+  bool OutputPic(H264Picture* pic);
+
+  State state_;
+
+  // A frame has been sent to hardware as the result of the last
+  // DecodeOneFrame() call.
+  bool frame_ready_at_hw_;
+
+  // Parser in use.
+  content::H264Parser parser_;
+
+  // DPB in use.
+  H264DPB dpb_;
+
+  // Picture currently being processed/decoded.
+  scoped_ptr<H264Picture> curr_pic_;
+
+  // Reference picture lists, constructed for each picture before decoding.
+  // Those lists are not owners of the pointers (DPB is).
+  H264Picture::PtrVector ref_pic_list0_;
+  H264Picture::PtrVector ref_pic_list1_;
+
+  // Global state values, needed in decoding. See spec.
+  int max_pic_order_cnt_lsb_;
+  int max_frame_num_;
+  int max_pic_num_;
+  int max_long_term_frame_idx_;
+
+  int frame_num_;
+  int prev_frame_num_;
+  int prev_frame_num_offset_;
+
+  // Values related to previously decoded reference picture.
+  bool prev_ref_has_memmgmnt5_;
+  int prev_ref_top_field_order_cnt_;
+  int prev_ref_pic_order_cnt_msb_;
+  int prev_ref_pic_order_cnt_lsb_;
+  H264Picture::Field prev_ref_field_;
+
+  // Currently active SPS and PPS.
+  int curr_sps_id_;
+  int curr_pps_id_;
+
+  // Output picture size.
+  int pic_width_;
+  int pic_height_;
+
+  // Data queued up for HW decoder, to be commited on next HW decode.
+  std::queue<VABufferID> pending_slice_bufs_;
+  std::queue<VABufferID> pending_va_bufs_;
+
+  // Manages binding of a client-provided output buffer (texture) to VASurface.
+  class DecodeSurface;
+
+  // Maps output_buffer_id to a decode surface. Used to look up surfaces
+  // on requests from the client.
+  typedef std::map<int32, linked_ptr<DecodeSurface> > DecodeSurfaces;
+  DecodeSurfaces decode_surfaces_;
+
+  // Number of decode surface currently available for decoding.
+  int num_available_decode_surfaces_;
+
+  // Maps decode surfaces to PicOrderCount, used to look up output buffers
+  // when a decision to output a picture has been made.
+  typedef std::map<int, DecodeSurface*> POCToDecodeSurfaces;
+  POCToDecodeSurfaces poc_to_decode_surfaces_;
+
+  // Find an available surface and assign it to given PicOrderCnt |poc|,
+  // removing it from the available surfaces pool. Return true if a surface
+  // has been found, false otherwise.
+  bool AssignSurfaceToPoC(int poc);
+
+  // Unassign a surface from |poc| and return a pointer to it, or NULL if there
+  // is no surface associated with given |poc|. Note that this does not make
+  // the surface available for reuse - as this can only happen after client
+  // returns the surface via ReusePictureBuffer() - but only removes its
+  // association with given |poc|.
+  DecodeSurface* UnassignSurfaceFromPoC(int poc);
+
+  // The id of current input buffer, which will be associated with an
+  // output picture if a frame is decoded successfully.
+  int32 curr_input_id_;
+
+  // X/GLX handles.
+  Display* x_display_;
+  GLXContext parent_glx_context_;
+  GLXFBConfig fb_config_;
+
+  // VA handles.
+  VADisplay va_display_;
+  VAConfigID va_config_id_;
+  VAContextID va_context_id_;
+  VAProfile profile_;
+
+  // Allocated VASurfaces.
+  VASurfaceID va_surface_ids_[kNumReqPictures];
+
+  // Called by decoder when a picture should be outputted.
+  OutputPicCallback output_pic_callback_;
+
+  DISALLOW_COPY_AND_ASSIGN(VaapiH264Decoder);
+};
+
+}  // namespace content
+
+#endif  // CONTENT_COMMON_GPU_MEDIA_VAAPI_H264_DECODER_H_
+