Youtube/Netflix integration to capture footprint of streaming #46
Comments
|
Is anyone working on that? I'd be interested in working on that one as a first contribution. |
|
I think you can go ahead!
What we'd need here is both the integration and a carbon model. Probably
one that shows the footprint of X min of watched netflix would be enough.
Thanks for the help ;-)
Oli
…On Mon, Jun 3, 2019 at 10:48 PM Valentin Iovene ***@***.***> wrote:
Is anyone working on that? I'd be interested in working on that one as a
first contribution.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#46>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAMUIKHBZRIYXRFUEYJ2FU3PYV7TDANCNFSM4HPUMPKQ>
.
--
Olivier Corradi
Founder, CEO
<https://www.linkedin.com/in/oliviercorradi> <https://twitter.com/corradio>
tmrow.com
|
|
The paper Evaluating Sustainable Interaction Design of Digital Services: The Case of YouTube analyses the carbon footprint of YouTube. It includes everything from the CDN (Content Delivery Network), the core and edge network (Internet infrastructure), residential access network / cellular network, end user device (phone or laptop). The result is computed for one year during which an estimated 1 billion hours of YouTube video are watched by users. The total electricity use is estimated to be 19.6 TWh. This corresponds (from the paper) to 10.1 MtCO2e for |
|
Wouldn't it make more sense to integrate with something like
https://www.rescuetime.com/ (or others?)
…On Tue, 4 Jun 2019 at 10:35, Valentin Iovene ***@***.***> wrote:
The paper Evaluating Sustainable Interaction Design of Digital Services:
The Case of YouTube <https://dl.acm.org/citation.cfm?id=3300627> analyses
the carbon footprint of YouTube. It includes everything from the CDN
(Content Delivery Network), the core and edge network (Internet
infrastructure), residential access network / cellular network, end user
device (phone or laptop). The result is computed for one year during which
an estimated 1 billion hours of YouTube video are watched by users. The
total electricity use is estimated to be 19.6 TWh. This corresponds (from
the paper) to 10.1 MtCO2e for 365 * 1e9 * 60 minutes of watching YouTube
videos, or ~0.45 gCO2e per minute of watching YouTube videos. From this
infographic
<https://climatecare.org/infographic-the-carbon-footprint-of-the-internet/>,
looking at a webpage with pictures or videos emits an estimated 0.2 gCO2e
per second. I feel like my 0.45 gCO2e per minute is pretty low. What do
you think @corradio <https://github.com/corradio>?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#46>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAT333VUYBOVXBUBRZ6ZEGDPYYSMVANCNFSM4HPUMPKQ>
.
|
|
Wow that's very low. For reference, my electricity consumption for a day is roughly ~1kg CO2. |
|
On 04.06.2019 01:44, martincollignon wrote:
Wouldn't it make more sense to integrate with something like
https://www.rescuetime.com/ (or others?)
Clearly, we want to monitor YouTube usage on all the user's devices.
For that, I would expect some YouTube official API that would allow
tomorrow to be linked to the user's YouTube account and access
information about videos watched by the user. If this API lets us
access how much time the user spent watching videos then we won. I
checked the [YouTube Data
API](https://developers.google.com/youtube/v3/docs/activities) and I
couldn't find that yet.
Your suggestion makes me think we could frame the problem in a more
general way. Instead of computing the impact of watching YouTube or
Netflix videos, maybe we want to estimate the carbon footprint for all
usages of electronic devices. People could have tomorrow software
running on each of their device (mobile phone, tablet, computer - and
possibly more devices later on, like cars). On each of these devices,
the tomorrow software monitors some metrics (CPU usage, how much data
is transferred through the network, ...) and estimates the carbon
footprint for that device. These footprints estimated for all the
user's devices are then sent server-side and aggregated.
…--
Valentin
|
|
Sounds like the ideal thing - and may be even better from a privacy
perspective? Is there a 3rd party tool which could be integrated that could
do that to avoid having to deal with building/maintaining extra software
from Tomorrow's side?
On Tue, 4 Jun 2019 at 12:59, Valentin Iovene <notifications@github.com>
wrote:
… On 04.06.2019 01:44, martincollignon wrote:
> Wouldn't it make more sense to integrate with something like
> https://www.rescuetime.com/ (or others?)
Clearly, we want to monitor YouTube usage on all the user's devices.
For that, I would expect some YouTube official API that would allow
tomorrow to be linked to the user's YouTube account and access
information about videos watched by the user. If this API lets us
access how much time the user spent watching videos then we won. I
checked the [YouTube Data
API](https://developers.google.com/youtube/v3/docs/activities) and I
couldn't find that yet.
Your suggestion makes me think we could frame the problem in a more
general way. Instead of computing the impact of watching YouTube or
Netflix videos, maybe we want to estimate the carbon footprint for all
usages of electronic devices. People could have tomorrow software
running on each of their device (mobile phone, tablet, computer - and
possibly more devices later on, like cars). On each of these devices,
the tomorrow software monitors some metrics (CPU usage, how much data
is transferred through the network, ...) and estimates the carbon
footprint for that device. These footprints estimated for all the
user's devices are then sent server-side and aggregated.
--
Valentin
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#46>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAT333QSINE7O2RZ6YHI2KDPYZDKZANCNFSM4HPUMPKQ>
.
|
|
On 04.06.2019 13:52, martincollignon wrote:
Sounds like the ideal thing - and may be even better from a privacy
perspective? Is there a 3rd party tool which could be integrated that
could do that to avoid having to deal with building/maintaining extra
software from Tomorrow's side?
I feel like we are asking ourselves a question that might be too broad.
Is Tomorrow aiming at using device-specific measures for estimating
carbon emissions, or aiming at connecting with 3rd parties that provide
external measures for estimating carbon emissions, or both at the same
time? Sounds like something only Tomorrow people can answer :-)
If my understanding is correct, the goal here is to answer the
following question: "how much carbon emission comes from my consumption
of videos?". From [1], YouTube corresponds to 11.4% of Internet's
worldwide downstream traffic while Netflix corresponds to 15%. By
covering YouTube and Netflix we cover ~25% of Internet's bandwidth:
this is **huge**. We can safely assume that by covering YouTube and
Netflix we cover *most* of the video consumption over the Internet. And
connecting to both these services is much simpler than implementing
device-specific software that tries to gather sometimes
hardware-specific metrics. The only thing we need is for the APIs of
these big companies to provide some measure of user consumption
(probably how many hours of videos were streamed to the user's device
is enough). Or, like you suggested, to have a 3rd party provider that
does that already for us. But I never heard of one of those.
[1] https://variety.com/2018/digital/news/netflix-15-percent-internet-bandwidth-worldwide-study-1202963207/
…--
Valentin
|
|
Hey guys,
It's indeed things we've thought about. To clarify: the monitoring of
electricity usage of devices is better covered by monitoring electricity
consumption in the home where you live.
In this particular case, it seems like we should connect to netflix/youtube
to figure out its footprint *excluding* the electricity consumption of your
device (which covered by your own smart meter).
I would at this stage verify whether or not the carbon intensity value we
got is realistic. It seems a bit low to me.
On Tue, Jun 4, 2019 at 4:16 PM Valentin Iovene <notifications@github.com>
wrote:
… On 04.06.2019 13:52, martincollignon wrote:
> Sounds like the ideal thing - and may be even better from a privacy
> perspective? Is there a 3rd party tool which could be integrated that
> could do that to avoid having to deal with building/maintaining extra
> software from Tomorrow's side?
I feel like we are asking ourselves a question that might be too broad.
Is Tomorrow aiming at using device-specific measures for estimating
carbon emissions, or aiming at connecting with 3rd parties that provide
external measures for estimating carbon emissions, or both at the same
time? Sounds like something only Tomorrow people can answer :-)
If my understanding is correct, the goal here is to answer the
following question: "how much carbon emission comes from my consumption
of videos?". From [1], YouTube corresponds to 11.4% of Internet's
worldwide downstream traffic while Netflix corresponds to 15%. By
covering YouTube and Netflix we cover ~25% of Internet's bandwidth:
this is **huge**. We can safely assume that by covering YouTube and
Netflix we cover *most* of the video consumption over the Internet. And
connecting to both these services is much simpler than implementing
device-specific software that tries to gather sometimes
hardware-specific metrics. The only thing we need is for the APIs of
these big companies to provide some measure of user consumption
(probably how many hours of videos were streamed to the user's device
is enough). Or, like you suggested, to have a 3rd party provider that
does that already for us. But I never heard of one of those.
[1]
https://variety.com/2018/digital/news/netflix-15-percent-internet-bandwidth-worldwide-study-1202963207/
--
Valentin
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#46>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAMUIKAYBXKH7P2UGVC34DLPYZ2NVANCNFSM4HPUMPKQ>
.
--
Olivier Corradi
Founder, CEO
<https://www.linkedin.com/in/oliviercorradi> <https://twitter.com/corradio>
tmrow.com
|
|
On 04.06.2019 07:23, Olivier Corradi wrote:
it seems like we should connect to netflix/youtube to figure out its
footprint *excluding* the electricity consumption of your device
(which covered by your own smart meter).
Makes total sense
I would at this stage verify whether or not the carbon intensity
value we got is realistic. It seems a bit low to me.
Yep, sent an email to the original authors of the paper I mentioned
earlier to get their input. Will keep you guys posted here if they
reply. Does it still seem low if we exclude the electricity consumption
of the device?
Also, should we account for the type of energy used by these companies
for their infrastructures? Greenpeace did a comparison [1] where
Google (and thus YouTube) scores very well (grade A) because they use
56% renewable while Amazon (and thus Netflix) scores very bad (grade D)
because they use only 17% renewable.
[1] https://www.greenpeace.org/archive-international/en/press/releases/2017/Amazon-still-lags-behind-Apple-Google-in-Greenpeace-renewable-energy-report/
…--
Valentin
|
|
https://www.emergeinteractive.com/insights/detail/does-irresponsible-web-development-contribute-to-global-warming/ (original source: http://aceee.org/files/proceedings/2012/data/papers/0193-000409.pdf)
that looks interesting! I think long term going for data downloaded is more
scalable and decently accurate than trying to plug-in to all services...
Then you just need to plug-in to telco for GB on data and router for at
home.
|
|
@corradio about #46 (comment) @tgv comment suggests 2 completely different figures (to be compared with your 1kgCO2eq/day household electricity use)
I think we should trust the 2019 peer-reviewed paper to start with. It's methodology seems transparent, and we can even subtract the "user-device" part based on the following breakdown from the paper That lowers our 0.45gCO2eq/min by 22.7%, to give 0.35gCO2eq/min of youtube streaming excluding end device electricity consumtion Note that they used IEA's GHG emission factors to convert TWh into tCO2eq, BUT also takes into account renewable energy purchase from google datacenter to reduce the carbon intensity for datacenters. I would recommend not to take these green certificates into account, but results will not change significantly (<5% difference) |
|
@brunolajoie Great, thanks! I got a response from the authors of the paper and they told me that my computation makes sense. |
|
I've created #56 |
|
Thank you everyone for the effort. This seems to be extremely low compared
to other activities we want to track (transportation, groceries etc..).
I therefore direct my attention to activities that represent larger
emissions, but if someone feels passionate about pushing this forward and
wants to build an integration that connects to youtube/netflix, please go
ahead! Seems like we have all the pieces.
Olivier
…On Wed, Jun 5, 2019 at 12:01 PM Valentin Iovene ***@***.***> wrote:
I've created #56 <#56>
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#46>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAMUIKFD2W46DZ5AY2LO6GLPY6FG7ANCNFSM4HPUMPKQ>
.
--
Olivier Corradi
Founder, CEO
<https://www.linkedin.com/in/oliviercorradi> <https://twitter.com/corradio>
tmrow.com
|
|
Some insights from the original authors of the paper
Hi Valentin,
Adding to Dan's comments -
Methodologically, given what you want, it is probably best to use the
overall energy figures we report for different segments of the
system, and then apply an emissions factor afterwards. We assume 10%
of the traffic uses the mobile network if I remember correctly (check
the paper and the appendix).
Also, importantly note the following: Our motivation was to determine
a *non-controversial lower bound* on YouTube emissions, to
demonstrate that the design approach we propose will result in
significant reductions even in this conservative case. (In other
research on streaming, not yet published, we instead assume a
distribution around a mean for the various variables.) In particular,
we assume a relatively low average resolution of video viewing.
(Again, see the paper for details) If people are using a higher
resolution than we assume - which nowadays is very common, but was
less so in 2016 which is the year we use data for - a good estimate
would be to pro-rata increase the network energy.
TBH, the data centres and core networks are very efficient, because
they are so highly utilised (mobile and home networking equipment
less so). The main factor increasing energy use / emissions
associated with streaming and other consumer services is probably
that we want it everywhere (and so more wifi and mobile deployment),
rather than that we want lots of it.
Best wishes
Chris Preist
On 06.06.2019 00:26, Olivier Corradi wrote:
Thank you everyone for the effort. This seems to be extremely low
compared to other activities we want to track (transportation,
groceries etc..). I therefore direct my attention to activities that
represent larger emissions, but if someone feels passionate about
pushing this forward and wants to build an integration that connects
to youtube/netflix, please go ahead! Seems like we have all the
pieces.
Yes, maybe building this integration is not worth it. I think we don't
really have all the pieces because thoroughly going through the YouTube
API I couldn't find some way to retrieve metrics for how much time the
user spent watching videos. We would have to do that differently.
Maybe I should focus on another integration that has more impact.
…--
Valentin
|
|
@martincollignon you mentioned this: This seems to be quite different numbers? Here it's more 18gCO2eq/minute! |
|
I found the scientific source behind this firefox extention:
Examining it carefully, the rate they compute is lower than 18g/min, but rather between 1 and 8. |
|
The full report is available here. Maybe the carbon footprint of your online video watching activity should be indexed on the energy mix of your country and of the country where the datacenter is located. This means it shouldn't be a single number for every user. |
|
Carbonalyzer sounds very interesting, but all data is only stored locally in the Browser so we won't get it into the app easily. However, it is an open source project with an MIT license, so probably there are ways to get it done. Do you think that having a possibility to import Carbonalyzer data into the Tomorrow app would be a good idea? |
|
Just note that Youtube is carbon neutral at the datacenter level, as Google either purchases green electricity or offsets the rests for 100% of all its operations. Read more at: https://sustainability.google/environment/:
And notably in this article. I am not sure if the tools estimating watching Youtube video take that into account when computing the CO2 from the used electricity. |
|
https://www.carbonbrief.org/factcheck-what-is-the-carbon-footprint-of-streaming-video-on-netflix Let's close this unless we can find APIs to pull web usage. |
|
@martincollignon Can we re-open this issue? |
|
Also youtube offers an API that let's you access to the user's history. Here is what I had noted at the time:
|
No description provided.
The text was updated successfully, but these errors were encountered: