Add media::VideoEncodeAccelerator with WebRTC integration

media/video/video_encode_accelerator.h

+// Copyright (c) 2013 The Chromium Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style license that can be
+// found in the LICENSE file.
+
+#ifndef MEDIA_VIDEO_VIDEO_ENCODE_ACCELERATOR_H_
+#define MEDIA_VIDEO_VIDEO_ENCODE_ACCELERATOR_H_
+
+#include <vector>
+
+#include "base/basictypes.h"
+#include "base/memory/ref_counted.h"
+#include "media/base/bitstream_buffer.h"
+#include "media/base/media_export.h"
+#include "media/base/video_decoder_config.h"
+#include "media/base/video_frame.h"
+
+namespace media {
+
+class BitstreamBuffer;
+class VideoFrame;
+
+// Video encoder interface.
+class MEDIA_EXPORT VideoEncodeAccelerator {
+ public:
+  virtual ~VideoEncodeAccelerator();
+
+  // Specification of an encoding profile supported by an encoder.
+  struct SupportedProfile {
+    VideoCodecProfile profile;
+    gfx::Size max_resolution;
+    struct {
+      uint32 numerator;
+      uint32 denominator;
+    } max_framerate;
+  };
+
+  // Enumeration of potential errors generated by the API.
+  enum Error {
+    // An operation was attempted during an incompatible encoder state.
+    kIllegalStateError,
+    // Invalid argument was passed to an API method.
+    kInvalidArgumentError,
+    // A failure occurred at the GPU process or one of its dependencies.
+    // Examples of such failures include GPU hardware failures, GPU driver
+    // failures, GPU library failures, GPU process programming errors, and so
+    // on.
+    kPlatformFailureError,
+  };
+
+  // Interface for clients that use VideoEncodeAccelerator.
+  class MEDIA_EXPORT Client {
+   public:
+    // Callback to notify client that encoder has been successfully initialized.
+    virtual void NotifyInitializeDone() = 0;
+
+    // Callback to tell the client what size of frames and buffers to provide
+    // for input and output.  The VEA disclaims use or ownership of all
+    // previously provided buffers once this callback is made.
+    // Parameters:
+    //  |input_count| is the number of input VideoFrames required for encoding.
+    //  The client should provide at least this many frames, since the encoder
+    //  may need to hold onto some subset of inputs as reference pictures.

[Ami GONE FROM CHROMIUM, 2013/08/05 18:44:38]

"provide at least this many frames" sounds like there's a
VEA::ProvideVideoFrame() call of some sort, which there isn't. 
I believe what really is is "encoding latency" - the number of frames that must
be fed to the encoder to guarantee that at least one frame's output is seen in a
bitstreambuffer (and subsequently that that frame is released).

Is this a param that could be returned in NotifyInitializeDone?  Because
returning it this late is going to be tricky to wire back to
VideoCaptureBufferPool in the browser process.

[sheu, 2013/08/06 06:16:36]

Done (comment)

RequireBitstreamBuffers() is called to return the input_count as soon as it can,
so I don't see how wiring it into NotifyInitializeDone() would help matters.  In
fact, in EVDA right now we return this callback immediately since the first
stage in our input pipe is GSC, which has no input_count limitations.

I'm thinking ahead to when we start optimizing the encode path, and I think for
something as simple as I420->NV12 swizzling it would possibly be more effective
to do this on the CPU with libyuv, during the copy step in GpuVEA.  In that
case, we'd be piping direct to MFC and then we would have to report the MFC
input_count limitation, which can't be queried until we have output buffers
queued by UseOuputBitstreamBuffer(), which can't happen until we return a
RequireBitstreamBuffers() -- yes it's a catch-22, but not one we have to deal
with presently, and in any case we're tracking this with the Exynos vendor
driver guys.

In any case, I think it makes sense to return NotifyInitializeDone() separately
from this.

+    //  |input_coded_size| is the logical size of the input frames (as reported
+    //  by VideoFrame::coded_size()) to encode, in pixels.  The encoder may have
+    //  hardware alignment requirements that make this different from
+    //  |input_visible_size|, as requested in Initialize(), in which case the
+    //  input VideoFrame to Encode() should be padded appropriately.

[Ami GONE FROM CHROMIUM, 2013/08/05 18:44:38]

I love this comment.  I wish all comments were like this.

+    //  |output_buffer_size| is the required size of output buffers for this
+    //  encoder in bytes.

[Ami GONE FROM CHROMIUM, 2013/08/05 18:44:38]

How does the client know how many buffers of this size to provide?

[sheu, 2013/08/06 06:16:36]

It can provide as many or few as it wants, > 0.

+    virtual void RequireBitstreamBuffers(int input_count,
+                                         const gfx::Size& input_coded_size,
+                                         size_t output_buffer_size) = 0;
+
+    // Callback to deliver encoded bitstream buffers.  The VEA disclaims use or
+    // ownership of the specified buffer once this callback is made.
+    // |bitstream_buffer_id| is the id of the buffer that is ready.
+    // |payload_size| is the byte size of the used portion of the buffer.
+    // |key_frame| is true if this delivered frame is a keyframe.

[Ami GONE FROM CHROMIUM, 2013/08/05 18:44:38]

I think you missed my comment:
On 2013/07/31 23:01:13, Ami Fischman wrote:

[sheu, 2013/08/06 06:16:36]

Done.

+    virtual void BitstreamBufferReady(int32 bitstream_buffer_id,
+                                      size_t payload_size,
+                                      bool key_frame) = 0;
+
+    // Error notification callback.
+    virtual void NotifyError(Error error) = 0;
+
+   protected:

[Ami GONE FROM CHROMIUM, 2013/08/05 18:44:38]

If you want to be providing a service here then make this private and add a
comment about why.  If you don't want to take the active voice about who should
be deleting the client then just remove the dtor.

[sheu, 2013/08/06 06:16:36]

Done.  Has to be protected though or nothing can inherit from it.

+    virtual ~Client() {}
+  };
+
+  // Video encoder functions.
+
+  // Initialize the video encoder with a specific configuration.  Called once
+  // per encoder construction.
+  // Parameters:
+  //  |input_format| is the frame format of the input stream (as would be
+  //  reported by VideoFrame::format() for frames passed to Encode()).

[Ami GONE FROM CHROMIUM, 2013/08/05 18:44:38]

I don't think that's right.  Here are two examples of webrtc encoder impls that
do handle mid-stream frame size changes:
https://code.google.com/p/chromium/codesearch#chromium/src/third_party/webrtc...
https://code.google.com/p/chromium/codesearch#chromium/src/third_party/webrtc...

[sheu, 2013/08/06 06:16:36]

Oh.  Hmm.  In that case we should just have RTCVideoEncoder tear down and
restart the encoder, since EVDA can't handle it anyways.

Can we worry about this a bit later?  pretty please

+  //  |input_visible_size| is the resolution of the input stream (as would be
+  //  reported by VideoFrame::visible_rect().size() for frames passed to
+  //  Encode()).
+  //  |output_profile| is the codec profile of the encoded output stream.
+  //  |initial_bitrate| is the initial bitrate of the encoded output stream,
+  //  in bits per second.
+  virtual void Initialize(media::VideoFrame::Format input_format,
+                          const gfx::Size& input_visible_size,
+                          VideoCodecProfile output_profile,
+                          int32 initial_bitrate) = 0;
+
+  // Encodes the given frame.
+  // Parameters:
+  //  |frame| is the VideoFrame that is to be encoded.
+  //  |force_keyframe| forces the encoding of a keyframe for this frame.
+  virtual void Encode(const scoped_refptr<VideoFrame>& frame,
+                      bool force_keyframe) = 0;
+
+  // Send a bitstream buffer to the encoder to be used for storing future
+  // encoded output.

[Ami GONE FROM CHROMIUM, 2013/08/05 18:44:38]

This sounds one-off'ish, but in reality must be called for each BB fed to
NotifyBitstreamBufferReady, right?
I think it would be useful to say that explicitly in this comment.

[sheu, 2013/08/06 06:16:36]

Done.

+  // Parameters:
+  //  |buffer| is the bitstream buffer to use for output.
+  virtual void UseOutputBitstreamBuffer(const BitstreamBuffer& buffer) = 0;
+
+  // Request a change to the encoding parameters.  This is only a request,
+  // fulfilled on a best-effort basis.
+  // Parameters:
+  //  |bitrate| is the requested new bitrate, in bits per second.
+  virtual void RequestEncodingParameterChange(int32 bitrate) = 0;
+
+  // Destroys the encoder: all pending inputs and outputs are dropped
+  // immediately and the component is freed.  This call may asynchornously free

[Ami GONE FROM CHROMIUM, 2013/08/05 18:44:38]

typo: asynchornously

[sheu, 2013/08/06 06:16:36]

Done.

+  // system resources, but its client-visible effects are synchronous.  After
+  // this method returns no more callbacks will be made on the client.  Deletes
+  // |this| unconditionally, so make sure to drop all pointers to it!
+  virtual void Destroy() = 0;
+};
+
+}  // namespace media
+
+#endif  // MEDIA_VIDEO_VIDEO_ENCODE_ACCELERATOR_H_