Your 3 stones of knowledge: the LIVErtising 2020 exam

March 25, 2020

Under the present dire circumstances, here is a confirmation of the exam format as I announced it during the first lecture, identical in format to the preceding years.

May 11 update: Check your Hubspot results in the relevant post on this blog.

May 1 update: Here are the Instructions for the oral exam conducted online by webcall on TEAMs – every student taking the exam must have read and understood these guidelines and apply them during the whole exam.

April 28 update: the school has just updated the exam schedule taking into account the need to keep distances – you’ll find the updated roster catering for an online exam by webcall – as we had anticipated this for the LIVErtising exam, this has a limited impact only on our exam – see below – specific information on the technical aspects coming on May 1.



The contents of the exam

The exams bears on all the different facets of the course, i.e.

  • the contents shared during each lecture based on the weekly keynotes and on the oral explanations (both IRL and online on TEAMs) reflected in your personal notes; I intend to share some videos where I focus on a few difficult items such as hypothesis testing, rephrasing the live videoconferences
  • the videos embedded in the slides; if you missed any, you’ll be happy to find (most of) them on the course Youtube channel
  • the chapter from Shoshana Zuboff’s book The Age of Surveillance Capitalism, which inspires the critical angle I took towards algorithmic advertising and more specifically Googlenomics
  • three quizzes I have tweeted to you to test your knowledge, which belong to the course contents – you can access them on this blog page
  • the different platforms, sites or services I invited you to test all along the course
  • your own research based on the course
  • April 22 update: video recording of some of the TEAMs lectures during quarantine, allowing for the technical mishaps

Your study must develop a solid understanding not only of these contents. It will also focus on the concepts that structure the course, and even more so the links between the different parts and their concepts. Do not base your study on the list below, which is only intended as a testing tool after you have done your study, not as a study basis.

The exam format

The evaluation itself will be oral – either IRL at my office (Etuve 703) or by videoconference if we are still homeridden. It will be based on three concepts from the course. You’ll draw three cards at random from a list including (1) core concepts (2) supporting concepts (3) illustrating concepts.

Here are some examples from the first lectures to give you an idea:

three stones of knowledge(1) core concepts: 4 models of communication, A/B testing, SOLOMO, mobile first, algorithmic communication, creativity vs data, engagement, facial recognition, KPI, reject H0, behavioural surplus, surveillance

(2) supporting concepts: cognitive bias, crosschannel approach, CTA, embed, Edward Bernays, Thorstein Veblen, heatmap, earned, IoT, NPS, wearables, Avinash Kaushik

(3) illustrating concepts: #EvianBottleService, ABTasty, American Red Cross, Apptimize, Bernard Marr, Echo Dot, Hotjar

I’ll display the complete list on May 6, soon after the last lecture.

April 22 update: here is the complete list for 2020 – I insist: these concepts are a testing tool, not a learning device. Do **not** start your study from the list as that would turn the course into a flat linear one-dimensional world, bringing you to lose sight of the essential: digital, and reality in general, is multi-dimensional, systematic… networked! Do you understand me?

Direct link to the file

Your task

Your aim will be to show you have clearly understood these concepts and how they relate to the evolution of marketing communication as we have analysed it in the course. In a nutshell, this evolution has taken us from

(1) interruptive/ top-down/ one-way/ outbound advertising over (2) permission/ bottom-up/ two-way/ inbound advertising to (3) networked/ collaborative/ social/ participatory/ connected communication (4) to reach a stage that is data-driven by algorithms, where AI is progressively taking over many aspects of marketing communication.

Each concept should be a door that opens up the global conceptual context it belongs to.

Additionally, your optional certifications with Hubspot Academy will prove very useful to extend your knowledge (I‘ll have published the results soon after the deadline, May 8 at midnight here – ) , and for those with a taste for even more, there is the information shared on Twitter with the #LIVErtising hashtag.

In a nutshell, your task will be to:

  • define those concepts
  • put them in their adequate conceptual context and develop that conceptual network
  • show how they connect with associated concepts
  • detail how they help us understand the evolution and specific digital dimension of marketing that the course focuses on

Make sure no language problem blurs your message – here is some revision work which may boost your mastery of some basic aspects of the language. I strongly encourage you to work through this regularly starting immediately.

This exam format is far more open than if you got three question about the course: it’s both easier and also more difficult. It gives you more possibilities: no danger of feeling stuck in a question you do not understand or remember – there is no single or expected answer, as long as it is relevant, information-rich, coherent and shows that you have understood the context. But is also gives you more responsibility: the need to build your own answer with all the bits and pieces of the course in order to provide a relevant reply that makes sense. In this respect the structure of your answer will be important.

Now, if you still have the feeling that your job consists of studying (possibly by heart) the definitions of a list of concepts, stop here and read the above explanations again, until you get rid of that feeling, because it is wrong!

I understand you may want to ask me questions about some of the above concepts. I invite you to submit all your questions as a comment at the bottom of this post. This will enable me to answer publicly and help you all. In earlier posts about the exam you’ll find my answers to such questions. You’ll already find an extensive Q&A list on the preceding posts announcing preceding exams.

The schedule

Here is the roster for the oral exam, considering conditions in Belgium enable us to take the exam face to face at IHECS on the scheduled days. Alternatively, it’ll be organised by webcall on a platform such as TEAMs or Skype – specific information will be provided on May 1. Some comments:

    • I have randomized the list to maximise fairness to all. Any two students may ask me to switch dates for any reason.
    • All Erasmus students have been registered on May 14, but any of them can move to Friday 29 just by letting me know. In case the exam on May 14 have to be taken online, I’ll be open to possibilities to take them offline as well on a date to be agreed on, probably Thursday June 18.
    • In case your name is on the list but you do not intend to present the exam, please let me know by mail asap. If your name is missing, let me know asap too!
    • As these changes will impact the whole roster, I’ll require everyone to check their time slot again later, definitely on the day preceding your exam.
    • April 28 updates: This is the definitive exam roster, catering for an individuel online exam by webcall, bar some minor changes due to circumstances
    • If the technical setup at your own place does not allow for an online exam by webcall, you must inform the school management and myself by May 4, to give us the opportunity to (try to) provide an alternative. I’ll investigate all such requests bearing in mind that a webcall exam must be considered as the (new) normal under these circumstances.

Meanwhile, let us take the opportunity to reconsider what is really important in life: health, love and friendship, healthy food, mutual help and humanity. Hoping this will inspire us more, again, later…

Like this? Share it!

37 Comments. Leave new

Rachel Andrianne
May 26, 2020 18:51

Hello Mr. Ranschaert,

I have two questions about the concepts:
1. Product listing: is it the company’s entire product portfolio or the Google shopping ads or featured snippets that appear on the SERP?
2. What is the link between attribution and google adwords?
3. Advertising media buying: If I understand correctly, once the SSP sends the request for an ad to the ad exchange, the ad exchange connects to the ad networks and DSP, which can both answer to the request, but just one can win the auction. Is that correct?
a. Is the ad network the middleman of the trading between media sales houses and media buying agencies?
b. What is the role of the media sales houses?
c. What is the real difference between ad networks and DSP?

Thank you in advance,


Hello Rachel,

1. I used the concept as I wanted to sketch the evolution of the SERP. In particular, I used the product listing concept to draw the attention on the disappearance of Google Ads (then they were still called Adwords) in the right rail on desktop i.e. on the right hand side, next to organic results. They have been replaced by elements such as the knowledge graph and product listings. Featured snippets are different. They refer to a different and more recent evolution in the SERP elements: quick snappy answers to a query, usually in the form of a definition, a list or a precise information element such as opening hours – they are now often followed by the “people also ask” SERP feature, above the organic listing. These featured snippet are not expected to generate a click.

2. These two concepts are not inherently connected and can be approached separately, but the Google ads platform enables the advertiser to implement more sophisticated attribution models. In a nutshell, attribution is the relative responsibility you attribute to each touchpoint of the CDJ in the eventual (i.e. final) conversion. Until recently, advertisers could do no better than a last click attribution model, as they had no data about the CDJ preceding the last-clicked ad leading to purchase. Digital marketing then made it possible to track the whole section of the CDJ that happened online, making it possible to determine the relative importance of each online touchpoint. Even more recently, big data leveraged both on- and off-line, merging our online and offline behaviour, even makes it possible to track the importance of most of the touchpoints, at least in theory and with the help of algorithms. As Google has access to virtually all aspects of our lives (cfr. behavioural surplus), using their advertising network will make it easier to produce attribution models that are pretty sophisticated and closer to reality.

3. Not completely: it comes closer to the actual working if you consider that the ad exchanges and the ad networks are two different business models, both algorithmic, using data to find a match between demand (produced by the media buying side, specifically the media buying agency and later the DSP) and supply (provided by the media sales house, later the SSP). The best match wins the deal (ad network) or the bid (ad exchange).
a. yes, indeed!
b. Aggregating inventory to package it up and offer it for sale to ad networks, and possibly ad exchanges – though in this case you can better consider they have evolved into an SSP
c. A DSP aggregates audiences and offers/supplies them packaged together in automated/algorithmic auctions happening in real time in the ad exchange, or more traditionally without real time bidding on an ad network. You need to picture this as companies providing the technical infrastructure making this possible, not as a company with staff trading ad space or time manually.

My advice to you, Rachel, is to understand the logic of the system – I have not had the time or possibility or wish to go into the workings of each individual component. But understanding the general workings of the system, understanding its philosophy, is the most useful starting point! To do so, it is key to see that the messy and complex system we have today actually results from a progressive insertion of a middlemen at each stage of the development.

I hope this helps 😉

Frederikke Amalie Naehr
May 12, 2020 08:28

Dear Jean-Pierre

I have some questions that I will like to ask you.
1) I know that the term ‘I’m 90% certain that version B will improve conversion’ is linked to A/B testing but I’m not sure what ‘90%’ stands for. Is it the number we need to be sure that version B is better?
2) I think that the term ‘H1, H2, H3, H4, H5’ is also linked to A/B testing but I have none notes about this concept?
3) The same applies with the two abbreviations ‘ANI’ and JEP’. I have searched the power points but can’t find anything on this two concepts?

I hope that you have time to answer my questions.
Thank you : )


Hello Frederikke,
let me point you in the right direction, hoping this helps.

1. we covered this in our discussion of A/B testing indeed, and this 90% certainty level relates to need for split tests to validate results with statistical hypothesis testing. Indeed, observing that version B performs better than version A with, say, an improved CTR of 12% is not enough. You absolutely need to test the result/difference on statistical relevance: you need to take the effect size and sample size into account and use appropriate stat test to reject the null hypothesis (H0) that states that there is no difference. As you cannot “prove” that H1 is true (i.e. that there is a real difference between A and B, not only in your sample, but also in every possible case, that is: “in the population”), all you can do is “disprove” or reject H0 with a specific certainty level, e.g. 90% or 95%, or if you are in medicine rather than in communication, 99% or 99.9%! I have fully developed this in slides 55 to 61 in the student slides!AkOFBhZJw3Ylw0IO6H9E-0xSibgY?e=GUR8VT and in the video call devoted to this. In a nutshell, I’d say that the notion is more closely linked to statistical hypothesis testing or statistical relevance testing, which is one step of A/B testing, and more generally multivariate testing.

2. I just suggested that “H0 vs H1” is connected to hypothesis testing. The series “H1, H2, H3…” on the other hand (without any H0) refers not to “hypothesis” but to “headline”: when writing the content of a webpage, you’ll organize and structure it carefully both for your readers and for the search engine crawlers, showing what is more important. One element of SEO technique to do so is the correct use of H1… tags.

3. ANI stands for Artificial Narrow Intelligence, the stage of development in machine intelligence which we have already reached today, preceding AGI and ASI. – JEP is the self-regulation body overseeing advertising in Belgium, Jury d’Ethique Publicitaire. One aspect this body regulates is the practice of influencer marketing. I understant the Danish equivalent is the Danish Consumer Ombudsman.

This should help you forward. For future questions, I suggest some of your fellow students try their hand at answering first, allowing me to monitor their answer. OK?

Julie Peksa
May 3, 2020 16:33

Hello Sir,

Here are my questions for the Q&A session of this Monday.

-On the 3Rs of influencer Marketing, can we say that the resonance is the world of mouth ?
-Does engagement rate detect of the Instagram account has bought fake followers ?
-Why do we say that 5G is scalable ? (compared to PWA ?)
-For the Net promoter score, from which figure are we a detractor ?
-Bounce rate: if you have 20%, why is it bad ? It doesn’t mean that there are few people who leave without interacting ?
-Is black hat legal in terms of competition ?

Thank you in advance!


Question answered orally in the last lecture.

Yasmine Ait Maskour
May 2, 2020 11:54

Hello sir,

I have some questions about the course,

First, is there 3Rs or 4Rs of Influencer Marketing. Indeed I found on the list of concepts that there were 4 but I noted only 3.
Second, I didn’t understand what N / A means in the types of social media engagement.

Third, what does RO.PO mean in the next sentence “The customer using SO / LO / MO communication is RO.PO = customers”

Finally, what does IRT mean in the following sentence “Your users expect you to react IRT”

Thank you in advance,


Question answered orally in the last lecture.

Margaux Snakkers
April 25, 2020 10:56

Hello Sir,

I have a few questions about the course and the concepts that I would like to ask during the Q&A session Monday.

First, sometimes a concept can refer to several chapters. For example, ‘’content’’ appears multiple times in the course. What are we supposed to do during the exam, then? Chose a specific chapter to develop the concept or develop all the possible applications of the concept?

Second, I didn’t understand the difference between ‘’link farming’’ and ‘’content farming’’. Does to refer to the same concept?

And lastly, what does PNB stand for?

Thank you in advance!

Charlotte Masure
March 31, 2020 11:11

Hello Sir,

I juste wanted to know if the comments and articles that you add outside of each slides in the PowerPoints must also be studied for the exam ? Thank you in advance


Thank you Charlotte, useful question. You are referring to the so-called presenter’s notes in the PPT version of the slides. They are only intended as a memo for me, they are unstructured, unedited and unorganized, definitely not intended for you to include in your study. As I move slides around, some are no longer correct or outdated. Now, if for a given slide, you happen to read the note and feel it helps you, OK. But there is no need to go through them!


Hello Sir,

There is a concept for which I can not find any information, it’s “What to listen for and who to listen to?”. We talked about the concept listening in the course but I couldn’t find this particular concept in my notes. Could you please help me?

Thank you!


Hello Merve,
do not think too far and take this literally – it is the title of one of the slides by the way: while you’re applying the listening attitude, what are elements you should focus your monitoring and research on (e.g. visual content featuring your logo) and who for (e.g. competitors).

katia de Le Hoye
June 10, 2019 09:54

Hello Sir,

I have 2 questions I would like to ask you.

I was wondering if you can explain a bit more the concept of #hijacking. I don’t understand if it’s the fact to use the same #than the brand, but in a wrong way (MyNYPD) or if it’s when you modify the brand # (McDiabetes). Or are they both #hijacking techniques?

Which kinds of TV are considered as addressable TV? I understood that digital TV and smart TV are but I’m not so sure.

Best regards,

Katia de Le Hoye


1. Both are examples of hashtag jacking, Katia, as they divert a brand hashtag from its original (branded) purpose.
2. Addressable TV uses targeting techniques and personalised data that enable the advertiser to deliver a specific ad to each household, which is not (yet) allowed in France for instance, but used in the US and in the pipeline in Belgium.

Morgane Dethier
June 10, 2019 09:14

Hello Sir,

One tiny detail is bothering me. I’ve written in my notes that the CTR (click through rate) is a metric. But we made a clear difference between metrics and KPI saying that the first ones were raw numbers and the other numbers put in a context such as rates, percentages, etc. Have I written it wrong or is there another explanation?
Thank you. Have a nice day


Hello Morgane,
Number of clicks is more of a metric, CTR more of a KPI, for the very reason you are giving. This means that the metric/KPI disctinction is to be considered in terms of a cline rather than a yes/no difference. Raw numbers are indeed at the metric end of the cline; more abstract realities being measured will qualify as KPIs and involve measures that are compounded.

Morgane Dethier
June 10, 2019 10:03

Got it ! Thanks.


Hello Sir,

I would like to know if you have a good definition to make difference between cross channel and omnichannel?


Hello Audrey,
cross channel communication integrates all communication channels in the consumer journey leading to conversion, while in omnichannel communication you manage to bring all channels in synergy by optimizing them simultaneously: they are no longer working alongside each other but conjointly.
What do you think?


Yes, super thank you! I


Hello sir,

I was wandering if you could explain the exact difference between branded podcasts and native podcasts by also giving an example for both. It would help picture the concepts more easily.
Thank you


This distinction between native and branded applies throughough marketing communication – you can refer to the notion of native advertising: advertising content that tries to blur the line separating editorial from advertising content. Native advertising adopts the look and feel of the editorial context in which it appears, trying to immerse itself in that context and go unnoticed as advertising matter. Much influencer marketing plays on this: advertising pretending to be editorial, toeing the thin ethical line separating honest advertising from advertising deception. That is something that regular advertising, clearly marked as advertising content, recognisable in its format, its timing, its place and time, its codes, … does not do – commercial pre-rolls on Youtube for example are clearly marked as such, or Google Ads.
Just apply this difference to podcasts: some are “branded”, in that they convey the brand they are marketing very clearly, for example on their own channel on social networks; others are native as they are piggybacking on existing content providers, adopting their style and editorial line, placing their own commercial message in the background, merging it with the non-commercial content.


It does. Thank you very much !

Sarah Vande Wyer
June 5, 2019 15:06

I would like to know to what is related the core concept “Intent” as it can be many things.

Best of day



Hello Sarah,
you can consider “intent” as specific advantage of search over other non-search digital actions: it enables marketers to what the searcher is looking for – their intent – or intention, if you prefer. This clearly positions SEM in the framework of inbound marketing. Search engines themselves are focusing their efforts on a better understanding of the searcher’s intent underlying their queries. This is where the Hummingbird update and the unfolding of AI in Rankbrain by Google play an essential role.
Does this help you on your way, Sarah?

Rosalie Schallon
June 4, 2019 20:05

Hello Sir,
I have a question about “Statistical significance” : I have understood that our results have to be statisticly significant when we are comparing 2 things. So, we have hypothesis 0 = the one we want to verify and hypothesis 1 = the contrary. I think results of my test are statisticly significant when they are in one of the small areas (at the extremity of the graph). Now, I don’t understand how to link it with an A/B test, how can we compare 2 different versions of a website page for example with such a graph ? What statisticly significant means concerning the AB/test ?

Thank you for you help!


Hello Rosalie,
you’re right to check your understanding of that concept, because it requires precision. I assume you realize that your short presentation of H0 vs H1, although correct, is very summarized, right?
Now, applying the notion of statistical significance to A/B testing, or multivariate testing too, takes your original version as the baseline. For example, using the picture of your product to illustrate your product page. In an A/B test process (situation analysis, planning / statement of hypothesis, executing / experiment set up, measurement / statistical analysis) you’ll formulate hypotheses to improve your conversion rate. For example, you have heard that pictures featuring people sell better than pictures of objects, so you want to try out using a picture of two adolescents using your product to illustrate the same product page. You now have the baseline, version A with a product picture, also called control version, and the test version, version B with the adolescent picture. Your next step will be to test which does a better job by gathering the respective conversion rates. After one business cycle you’ll compare the results statistically, i.e. you’ll not only look at the difference (say, version A got 9 conversions from 120 visitors, version B got 16 conversions) to determine if the growth in conversions is sufficiently big to be arithmetically or statistically significant, you’ll also run a statistical test – you can use the tools I suggested or this one – such a test will determine, comparing the theoretical distribution of conversions in version A and in version B, if the observed difference is due to chance (that is the no-difference hypothesis called H0) as it is not big enough on the basis of the number of observations, or if the observed difference is big enough to appear theoretically with a degree of probability that is so low (for example in only 5% of the theoretical distribution) that we agree it is **not** due to chance but to a real difference (this is H1, which we consider as validated with that very level of probability).
To express the same reasoning differently: the hypothesis of no-difference H0 is not rejected as long as the difference in conversion rates is not extreme enough to consider that the results belong to two different distribution curves, but that they actually belong to the same distribution and that the difference is due to chance only; as soon as the difference is big enough to appear with a low probability (the p-value, say 5%), you are going to consider that the results you got are too different to belong to the same distribution, so you’ll reject the null hypothesis with a degree of certainty of n% (also called confidence level). You never prove that H1 is correct, that is to say you never prove that there is a real difference; you can only reject the no-difference hypothesis with a certain degree of certainty. That degree of certainty is the risk you take of stating that H0 is rejected while it is actually true.
Now, depending on the nature of the data you compare (in an A/B test is the number of conversions), you’ll have to use a specific statistical test, which varies depending on the type of value. But the general reasoning applies in all those cases. Using the A/B test significance calculators we have mentioned prevents you from having to chose which test is adapted to A/B test. There are also other elements to take into account if you want to use statistics for scientific research, which we have completely overlooked, such as the power of a test.
I hope this helps!


Thank you so much!


+ I’m also confused about the term “FTC”. I know it stands for Federal Trade Commission, it protects consumers and prevent from anticompetitive business practices, but I don’t know at what part of the course we must connect that? Is it right to say that the FTC tackles ad’fraud? I’ve read that the FTC fights against advertising fraud but I think it’s about misleading advertising and not necessarily ad’fraud as we seen it in class (click fraud, …) ? Thank you 🙂


I like your question, Marie, because it enables me to clear up that concept while at the same time illustrating my hopes for the exam.
We met the FTC in our discussion of influencer marketing. One downside is indeed the fraud this generates. A major advertiser, Unilever, reacted fiercely against this at the end of 2018 in the world press. This has forced both the authorities and the advertising industry to tackle the problem. In the USA the FTC, whose job is to protect America’s consumers, has issued their FTC Rules for Influencers.
These are the basic elements to address the FTC concept as we saw it in LIVErtising 19.02 .
Now, this opens a network of connections that you are invited to explore. The more you do this, the better. Those connected concepts include:
– influencer marketing in general, of course, but more specifically how influencer fraud is addressed in Belgium (the JEP’s Recommandations en Matière de Marketing d’Influence), i.e. through self-regulation rather than law – this may connect to the IAB, which is the self-regulatory body covering so-called interactive, i.e. digital marketing communication;
– fraud in digital marketing more generally, which plagues different types of digital advertising, click fraud for instance; the causes of these frauds, the mechanisms, their impacts – for example the fact that this feeds the growing fake news concerns; also their solutions, one of which may include the use of the block chain (cfr the video about blockchain advertising).
This is not to say that you must include all these concepts in your answer, but that you can start from one concept, even a detailed one, to explore its connection network.
All right? Let me know!


Thank you so much for your answer 🙂 it does help !


Hello sir,

I’ve question about the term “We are Social Singapore” in the list. A lot of statistics in the course come from “we are social”, but not necessarily of Singapore. Is this a particular reason you added “singapore” in the list? I’m a little confused about that. Thank you 🙂


Hello Marie,
you are right, many slides with data about digital are branded “we are social”, with a repository you can find at . We Are Social is a global agency with offices all around the world. The Singapore offices have taken the lead in producing the reports sketching the state of digital at regular intervals, but Singapore is progressively dropped in references to the reports.
So, your question is legitimate but spots a point of detail.


A question that has just come in through mail: “I am writing to you concerning the term “mars” in the livertising vocabulary list. I truly cannot find it’s true meaning except for the fact that it is also a biscuit which is definitely not the correct answer. Could you help me understand what it is ? I will transfer the answer to the other students who are also stuck with this particular word/concept.”


Thank you Leïla. No help mailing me your questions, please ask them as a comment added to this blog post, which enables everybody to read both my and your peers’ answers.
Now, Mars! I remember adding this after sending the students’ keynote, so here is the slide: – does that help? Gauthier drilled down my own analysis by observing the branded box containing the Mars bars.
Let me know, OK?


Thank you very much, that does help !


Leave a Reply

Your email address will not be published.