Dynamics of niche creation in sponsored search markets

Abstract

Traditional advertising mediums such as TV, Radio, and Print were broadcast based affording access to large audience(s). The unit of buying an advertisement slot (time or space) was based on the 'mass' that the medium could reach. As a result, advertising investments comprised high fixed costs. In contrast, new age online advertising markets such as advertising markets on search engines, enabled by information and communication technology, allowed reaching out to individual users surfing the web using a search query or keyword. On search engines, advertisers only pay for clicks on their ads. Consequently, the nature of investment in advertising changed from one of high fixed cost associated with reaching out to a 'mass' of users to that of variable cost associated with a 'click' on the ad by 'a user'. These changes suited the needs of smaller advertisers by creating accessible advertising markets. Thus, one would have expected a fragmented market structure in online advertising markets. However, fragments of available empirical evidence point towards high concentration in search engine advertising markets (SSMs). This research enquirers into these seemingly contradictory observations wherein the micro design of individual transaction is access friendly but the market at an aggregate level (all transactions) exhibits high concentration. This research is an endeavour to theoretically understand SSMs from a systemic perspective while remaining close to real life advertiser practices gleaned through an ethnographic immersion. Towards this end, this research uses an agent-based model that employs a novel way to integrate the structure of semantic relationship between search queries, captures the real life advertiser practices of account management, and incorporates the rules and mechanisms used by the search engine to govern the market. The agent-based model built, integrates various strands of literature in SSMs and enables setting up of experiments that shed light on a systemic understanding of the market. With an over-arching goal of understanding how the systemic characteristic of high concentration emerges in SSMs, where the design of individual transaction appears to enable and ease entry, this thesis makes three major enquiries. The first enquiry takes a close look at algorithmic aspects that impose the rules of the market. One such element of the algorithm is the computation of relevance score for different advertisers on a keyword to be used as a parameter to rank ads. One of the constructs used by a search engine in computing this relevance score is the past click-through-rate (CTR) of an advertiser on a particular search query. Literature suggests that doing so rewards an advertiser with past success on a keyword generating rich-gets-richer dynamics on an individual keyword. However, how does one explain the diffusion of rich-gets-richer dynamics (increasing returns) associated with individual keyword to become a market-wide phenomenon, given the fact that these rich-gets-richer dynamics can be overturned by higher bids from other advertisers? An agent-based model allows us to study the temporal evolution of the simulated sponsored search market and understand concentration as an emergent characteristic of the market. We extend the stream of work on increasing returns in sponsored search markets and show that increasing return dynamics at an individual keyword coupled with the advertiser practice of managing 'keywords as a portfolio' [with a cap on maximum price paid per click operating at the portfolio level] can endogenously make sponsored search markets concentrated. Simulating counterfactual markets with different time windows used for CTR computation explicates the complex relationship between window size used to compute CTR, the strength of increasing return dynamics, and the optimization practices of the advertisers. It is found that market concentration has a counter-intuitive non-linear (inverted U) relationship with the size of the time window used for CTR computation. This is particularly interesting because counter to the observed results, both intuition and theory of increasing returns would suggest that as the size of window is increased the strength of reinforcing dynamics would be stronger and a higher concentration would be observed. The second enquiry looks at sponsored search markets from the lens of market or resource partitioning theory to understand how specialist advertisers thrive even when the market shows high concentration. Unlike the mechanisms of market partitioning proposed in extant theory which have their basis on capacity utilization and apportioning of fixed costs, market partitioning in SSMs arises from exploitative economies that have their basis in increasing return dynamics through temporally accumulated clicks (cumulative scale) on individual keywords. Bid optimization with cost cap on the portfolio allows for conversion of economies of cumulative scale of a generalist to economies of scope that could be used to displace specialists in their (specialists') niche. However, this ability to convert available economies of scale to additional economies of scope by an individual advertiser is contingent on inter and intra-category competitive intensity which is sensitive to the subpopulation mix of the competing advertisers. The market displays maximum concentration for a subpopulation configuration where the specialists successfully garner resources around their niche, showing that high market concentration and increased access of specialist advertisers is not contradictory, but, endogenous to the design and mechanism of SSMs. The third enquiry tries to build a systemic understanding of search engine revenue which has so far only been looked at from the perspective of an individual auction. It focuses on understanding the implications of 'local-rule-following' behaviour of heterogeneous advertiser groups on advertiser costs and search engine revenue. Moving away from the widely used assumption of 'known value per click' in the SSM literature and basing our analysis on optimization practices captured through an ethnographic study in an advertising agency it was found that search engine 'revenue' has systemic properties. Average revenue from an individual keyword is an emergent outcome that depends on advertiser heterogeneity and optimization heuristics used. From an advertiser's perspective, the choice of an optimization heuristic and the subpopulation-mix of competing advertisers have non-trivial implications on the price charged per click. Adopting an agent based approach allows market analysis from two different levels (search engine and search queries), from the perspective of different stakeholders (search engine and advertiser) and, at the same time makes the model a closer depiction of reality in comparison with extant analytical models.It may be emphasized that, by addressing a broad range of questions within the domain of search engine advertising, this research demonstrates the salience of using an agent-based simulations approach in enhancing our understanding of these markets. The agent-based model allows, (1) exploring the more complex and realistic setting comprising N-advertiser and M-keywords, thereby, departing from the dominant single advertiser (or single keyword) approach used in analytical models that dominate existing literature, (2) relational modelling of keywords that helps relax the assumption of independent valuation of keywords, and (3) modelling 'rule-following' advertisers that dynamically update bids based on heuristics, thus, moving beyond the assumption of 'known keyword value'. These shifts allow for a closer representation of the real market. Such a model allows analysis along different dimensions which include search query valuation, cost analysis for an individual advertiser, search engine's revenue as an outcome of different optimization heuristics, click shares among different classes of advertisers, and distribution of clicks of all advertisers at an aggregate level. The ability of agent-based models to perform counterfactual 'what-if' simulations contributes towards theory building by affording a nuanced understanding of the phenomenon. The three components taken together provide a holistic view of sponsored search markets where heterogeneous advertisers having different scope of operations, following different bidding heuristics compete against each other for clicks on their ads under the pricing and ranking rules imposed by the search engine's algorithm.