Towards A Unified Marketing Optimization Framework For E Commerce Sponsored Search

Abstract

As the largest e-commerce platform, Taobao helps advertisers reach billions of search queries each day via sponsored search, which has also contributed considerable revenue to the platform. An efficient bidding strategy to cater to diverse advertiser demands while balancing platform revenue and consumer experience is significant to a healthy and sustainable marketing ecosystem. In this paper we propose Customer Intelligent Agent (CIA), a bidding optimization framework which implements an impression-level bidding to reflect advertisers’ conversion willingness and budget control. In this way, CIA is capable of fulfilling various e-commerce advertiser demands on different levels, such as Gross Merchandise Volume optimization, style comparison etc. Additionally, a replay based simulation system is designed to predict the performance of different take-rate. CIA unifies the benefits of three parties in the marketing ecosystem without changing the Generalized Second Price mechanism. Our extensive offline simulations and large-scale online experiments on Taobao Search Advertising (TSA) platform verify the high effectiveness of the CIA framework. Moreover, CIA has been deployed online as a major bidding tool in TSA.

1 Introduction

Sponsored search has provided considerable revenue for universal search engines such as Google, Bing and vertical e-commerce websites like Taobao and Amazon. The 2017 Tmall Double Eleven shopping festival sees total sales of 25.4 billion (Reuters 2017), showing the prosperity of online shopping and associated advertising market. As the biggest e-commerce platform in China (Edquid 2017), more than 300 million consumers in Taobao reach commodities via keyword search each day, bringing daily 10 billion search queries and subsequent page views (PVs). Such massive PVs provide advertisers (mainly Taobao sellers) sufficient opportunities to expose their commodities and services.

Taobao Search Advertising (TSA) is a suite of commercial advertising service 1 . Each advertiser maintains a treelike TSA account as shown in Fig. 1. The account has a balance and consists of several budget-limited marketing campaigns, while a campaign accommodates a set of ADs to be promoted. At the most fine-grained level, each AD

Figure 1: Advertiser TSA account architecture.

specifies multiple relevant keywords and sets a fixed bid on each of them to compete for impressions. Search queries initiated by consumers retrieve relevant ADs through exact or broad keyword matching. Generally, all the ADs are ranked by the product of bid and predicted Click Through Rate (CTR). Once clicked, they will be charged the minimal price to maintain their position (Generalized Second Price, GSP), to maximize the expected revenue of the platform (King, Atkins, and Schwarz 2007). Cumulative postclick behaviors such as adding to collections/shopping carts and making purchases are used as ranking signals in future organic search, in which way TSA is especially useful for the cold start of new arrivals and earning persistent revenue.

Apart from keyword selection, advertisers can optimize their marketing demands in TSA by way of cautiously determining strategic bids. Nevertheless, such optimization is intractable due to the shortage of auction competition information, computation, and storage capacity. On the contrary, the advertising platform is omniscient and amenable to help advertisers bid for a healthy marketing ecology. However, if the platform takes the duty to design a bidding strategy for advertisers, new challenges might appear. Firstly, under the Pay Per Click (PPC) pricing mechanism, a mismatch of profits naturally arises between the platform’s immediate revenue and advertiser’s Gross Merchandise Volume (GMV), i.e., the platform charges the click while most advertisers bid for the conversion/purchase. Secondly, any bidding strategy designed by the platform is inevitable to be faced with the multi-agent problem, i.e., an uncertain amount of heterogeneous advertisers might opt in proxy bidding, which makes the strategy difficult to optimize. Last but not least, millions of varied-sized advertisers hosted

In TSA, an advertising commodity or service is called an AD.

by TSA have quite different marketing demands, showing diverse preferences in impression-level bidding. Existing work mostly falls into the maximization of a single advertiser’s profit in display advertising by the Demand Side Platforms (DSPs, e.g., iPinYou (Ren et al. 2017; Cai et al. 2017; Zhang, Yuan, and Wang 2014), JD (Wang et al. 2017), and Media6Degrees (Claudia et al. 2012)), and are not motivated to take platform revenue and consumer experience into account, which is critical for an advertising platform.

In this paper, we cater to various advertiser demands with a unified impression-level bidding based demand optimization framework called Customer Intelligent Agent (CIA). CIA keeps advertisers’ knobs on keyword-level bids to convey varied-level Return-On-Investment (ROI) expectations. Meanwhile, the impression-level bid is applied considering both real-time AD-query features and advertisers’ demandspecific ROI preferences. Combined with CTR and CVR (Conversion Rate) prediction modules (Gai et al. 2017), CIA balances the benefits of three parties without changing the original auction mechanism. A replay based simulation system takes the historical auction in to evaluate the winning probability and daily performance of different bids as the optimization tool. Offline simulations and online A/B test show that it outperforms optimized fixed keyword-level bids for various demands fulfilling. CIA based GMV optimization and style comparison have been widely adopted by advertisers 3 in the TSA platform.

The rest of this paper is organized as follows. In Section 2 we give a more detailed analysis of related work on bidding optimization. The architecture of CIA is given in Section 3 where we formulate the optimization problems with their constraints. Section 4 discusses the influence of CIA bidding in the GSP mechanism. And we give a replay based implementation of CIA in Section 5. In Section 6 we report some simulation and experimental results illustrating the performance of CIA on the TSA system. Section 7 concludes the paper.

2 Related work

Most existing bidding strategies designed by DSPs aim to optimize only one single advertiser’s profits by assuming that the competitive environment is stationary (Diemert et al. 2017; Ren et al. 2017; Zhang, Yuan, and Wang 2014; Claudia et al. 2012). In the case of unconstrained budget, truthful bidding in GSP has proved to achieve the Nash-Equilibrium for a single item auction (Krishna 2009). In (Zhang, Yuan, and Wang 2014), Zhang et al. studied the optimal bidding in display advertising with budget constraints on sequential bids, with the expected number of clicks maximized. Relying on independent CTR and bid landscape prediction modules, the authors proved that concave bidders paying more attention to low CTR impressions might be better. Later, if the budget was directly on the charging price, Zhang et al. proved that linear bidding is optimal (Ren et al. 2017) and a joint optimization including CTR estimation, market price prediction and bidding were applied. However, all these methods consider only a single advertiser’s profits. Moreover, a reasonable budget constraint is usually not directly available in TSA practice.

Other novel work also exists in display advertising. Considering the sequential decision processes in impression bidding, Cai et al. utilized reinforcement learning to bid (Cai et al. 2017). With the assumption of i.i.d impression features, they derived an optimal real-time bidder under budget and auction volume constraints. A similar idea was also used in JD DSP business, with deep Q-learner in its bidder(Wang et al. 2017). Reinforcement learning is a straightforward solution to the scenario of delayed rewards in online advertising. Diemert et al. improved single-impression bidder by attributing post-impression actions into previous bids to decouple the sequential bidding process (Diemert et al. 2017). Another related line of bidding optimization research lies in smart pacing, which helps budget-limited advertisers decide whether to bid or an impression or not (Karande, Mehta, and Srikant 2013; Lee, Jalali, and Dasdan 2013).

In sponsored search, the work on joint optimization of keyword-level bids and budget allocation over a campaign (several campaigns shared a budget) (Zhang et al. 2012) resembles our scenario. More keyword-level sponsored search bidding work can be found in (Qin, Chen, and Liu 2015). However, keyword-level bidding does not distinguish impressions from the same query. Such coarse-grained bidding is incapable of serving advertisers in an optimal way or meeting the ROI constraints well.

3 CIA Framework in Taobao Sponsored Search

Marketing demands at TSA can be classified into two categories, i.e., basic and compound demands. Basic demands include acquisitions of impressions, clicks and conversions, which are ubiquitous in online advertising. Particularly, for e-commerce platforms like Taobao, compound demands are more platform-specific, which cover the complete marketing cycle of an AD. Detailed demands and their Key-Performance-Indicators (KPIs) are summarized in Tab. 1.

Table 1: Various advertiser demands and KPIs in TSA

Demands	Description	KPI
	Impression	Number of impressions
Basic	Click	Number of clicks
	Conversion	Collections/Shopping carts/GMV
	Style comparison	Fast comparison of different styles under a budget
Compound	Audience expansion	GMV from new consumers
	Price-off promotions	Fast clearance with an acceptable ROI

Fig. 2 illustrates how CIA framework works within the TSA system. Advertisers express their marketing demands on different ADs via keyword-level bids or direct AD-level take-rate (abbreviated as tk) and CIA optimizes impressionlevel bids respecting the demands. Optimizer balances the take-rate and advertising costs. A replay based simulation system is utilized to obtain the solution. CTR/CVR prediction models are also necessary to characterize an impression

³More than 50% of all the ADs switched to CIA bidding, contributing over 30% of the daily revenue according to the statistics in Feb. 2018.

but will not be elaborated in detail as they are not the key contributions of our work. All the competitive ADs and their contextual information in an auction are recorded in logs for the replay system.

Advertisers’ demands are optimized under CIA in equation (1). Regardless of various demands, a reasonable budget constraint is necessary. Additionally, advertisers also bid with the expectation of conversions (adding to collections/shopping carts/purchase) in e-commerce. Each dollar spent should account for some returns, only high or low for different demands. Therefore, demand optimization should also take unit conversion cost (aka tk) as a constraint.

$\max_{bid}$ KPI

$s.t.$ budget constraint, (1)

unit conversion cost constraint.

While optimization objectives are exactly demonstrated in Tab. 1, the constraints, despite roughly depicted above, are still challenging to be specified. Firstly, budget constraints are set only at the campaign level in TSA, while AD-level tk to regulate impression-level bidding requires a reasonable AD-level budget. Secondly, advertisers prefer to use fixed keyword-level bids instead of tk to express their expected unit conversion cost due to business secrets or inaccurate CTR predictions. To sum up, we have to infer reasonable tk and also budget from keyword-level bid settings. We will elaborate on the inference in the following section.

3.1 Bridging keyword-level bid settings and constraints

Generally, advertisers choose the keywords of an AD and set fixed bids. Given m auctions (j=1,…,m) which involves the AD, the accumulated ROI can be calculated as:

$R = \frac{\sum_{j=1}^{m} ctr_{j} * cvr_{j} * ip_{j}}{\sum_{j=1}^{m} ctr_{j} * c_{j}} \approx \frac{\sum_{j=1}^{m} ctr_{j} * cvr_{j} * ip_{j}}{\sum_{j=1}^{m} ctr_{j} * b_{j}}, \quad (2)$

where $c_j$ is the click cost and can be approximated with the corresponding keyword bid $b_j$ to obtain a lower bound of ROI while ip stands for the item price. Following the above formula, we actually obtain an expected ROI from the original bid settings and auction log. Implicitly, this ROI reflects the advertiser’s implicit rationality in the marketing environment.

Meanwhile, it is sufficient to guarantee the cumulative ROI if each advertiser bids with an acceptable ROI in each auction. An impression-level roi satisfies that,

$roi = \frac{ctr * cvr * ip}{ctr * c},$ (3)

where c is the cost upon clicking. Let $roi \geq R$ , then $c \leq \frac{cvr*ip}{R}$ . In GSP it usually satisfies that $c \leq bid$ , therefore it is sufficient to bid as,

$bid = tk * cvr * ip, \quad tk = \frac{1}{R}.$ (4)

Furthermore, the advertiser’s changing keyword-level bids can easily propagate to tk given the historical auction log.

$\Delta tk \approx \frac{\sum_{j=1}^{m} ctr_j * \Delta b_j}{\sum_{j=1}^{m} ctr_j * cvr_j * ip_j},$ (5)

where $\Delta(\cdot)$ corresponds to the perturbations on its input argument.

In the above analysis, we take the auction log of an AD, arriving an AD-level tk. Campaign/keyword-level tks can be derived likewise with corresponding logs. tk can be updated daily with historical auction logs or in an online way from real-time auctions. In the latter way, real-time changes in advertisers’ keyword-level bid settings and auction distribution can be easily captured. In this paper, we take the AD-level tk for the daily distribution of an AD’s auctions is relatively stationary (which will be justified in Section 5), and also for its high-interpretability to advertisers.

Relying on the historical auction log, a reasonable AD-level virtual budget can also be obtained as:

$B = \sum_{j=1}^{m} ctr_j * b_j.$ (6)

If the marketing environment is stationary across days, we believe that such a budget shall be fair and sufficient for the advertiser. In this way, we do not need to explicitly split the campaign budget into each AD.

To optimize various campaign-level demands, CIA introduces an additional $cost\ regulation\ factor\ \alpha$ to adjust AD-level tk in a campaign, leading to a generalized bidding formula:

bid = \alpha * tk * cvr * ip. \tag{7}

Generally, an $\alpha$ in the range of (0,1] can roughly guarantee that the advertiser will gain a higher ROI than previous keyword-level bidding. However, for the approximations used, the de facto effective $\alpha$ might be larger than 1. Additionally, different demands might lead to a quite different feasible range of $\alpha$ . For each AD in a campaign, query distribution is quite different from each other but shares a common budget. For various demands, this leaves us space to optimize different objectives by regulating AD-level $\alpha$ .

3.2 Demands optimization

An AD-level tk simplifies advertisers’ bidding control. Meanwhile, campaign-level demand is the most natural way for advertisers to manage. To this end, in this section we tackle the two most representative demands of advertisers, i.e., campaign-level GMV (conversion) optimization and style comparison by regulating AD-level tk (aka, $\alpha$ ) to illustrate the whole optimization process of CIA framework.

In the following formulation, for a campaign with n ADs (indexed by i=1,…n), we denote the expected daily GMV and cost of ADs if bidding with CIA as $\mathbf{y}, \mathbf{z} \in \mathbb{R}^n$ . Under the same auction environment, corresponding keyword-level bids will lead to a cost of $\tilde{\mathbf{z}}$ . And the cost regulation vector is denoted as $\alpha \in \mathbb{R}^n_+$ . Beyond standard keyword-level

Figure 2: Typical query processing procedure via CIA within TSA.

bids, advertiser’s bids of the n ADs lies in the range of $[\mathbf{l}, \mathbf{u}]$ , $\mathbf{l}, \mathbf{u} \in \mathbb{R}_+^n$ . 1 represents the n-dimensional column vector with all its elements to be one. $\alpha_i$ is the cost regulation factor of the tk for the i-th AD. The same subscript logic applies to $\mathbf{y}$ and $\mathbf{z}, \mathbf{x}^T$ denotes the transpose of $\mathbf{x}, ||\cdot||_2$ is the Euclidean norm of the input argument.

Advertiser campaign GMV optimization Campaign-level GMV optimization is fundamental for advertisers in TSA. For a campaign, we want to determine each AD’s $\alpha_i$ so that GMV is maximized. For a campaign-level GMV, the problem can be formulated as:

$\max_{\boldsymbol{\alpha}} \quad \mathbf{1}^{T} \mathbf{y}$

$s.t. \quad (\beta - \epsilon) \mathbf{1}^{T} \tilde{\mathbf{z}} \leq \mathbf{1}^{T} \mathbf{z} \leq (\beta + \epsilon) \mathbf{1}^{T} \tilde{\mathbf{z}}, \qquad (8)$

$\beta \in (0, 1], \epsilon \in [0, \beta),$

where $\beta$ is the cost regulation parameter and $\epsilon$ is the permissible offset determined from the demands. In practice, $\beta$ is set no larger than 1 to ensure that the cost does not exceed the keyword bidding cost, i.e., the virtual budget. Meanwhile $\alpha$ is constrained in ranges determined from advertiser’s tolerable ranges of $[1, \mathbf{u}]$ .

CIA generates fine-grained impression-level bids with coarse-grained AD-level $\alpha$ and tk for implicit ROI constraints. Note that in basic CIA, $\alpha_i \leq 1$ is used to guarantee a higher ROI while such constraint on $\alpha_i$ is eliminated but replaced with the ROI range implied by $[l_i, u_i]$ in the GMV optimization. To solve the problem, we first utilize the bid ranges $[\mathbf{l}, \mathbf{u}]$ to derive a feasible range of each $\alpha_i$ . For this purpose, firstly $l_i$ and $u_i$ are replayed as keyword-level bids to obtain the corresponding cost range of the i-th AD. This is reasonable since the cost is monotonically non-increasing with the bids. Then the replay module is inversely applied to approximate the feasible range of $\alpha_i$ from cost.

Assume there is a set of k effective $\alpha_i$ , denoted as $\{\alpha_{i1},...,\alpha_{ik}\}^4$ . For each AD, $\alpha_{ij}(j=1,...,k)$ is eval-

uated by the replay module to yield a valuation point $(\alpha_{ij}, y_{ij}, z_{ij})$ . $\tilde{z}_i$ is obtained by replaying with keyword-level bids. Following the discretization, the GMV optimization problem can be reformulated as follows:

$\max_{\{x_{ij}\}} \sum_{i=1}^{n} \sum_{j=1}^{k} x_{ij} y_{ij}$ s.t. $\sum_{j=1}^{k} x_{ij} = 1, x_{ij} \in \{0, 1\}, \forall i = 1, ..., n,$

$(\beta - \epsilon) \sum_{i=1}^{n} \tilde{z}_{i} \leq \sum_{i=1}^{n} \sum_{j=1}^{k} x_{ij} z_{ij} \leq (\beta + \epsilon) \sum_{i=1}^{n} \tilde{z}_{i}$ (9)

where $x_{ij}$ is an indicator variable that $x_{ij} = 1$ means $\alpha_{ij}$ is selected for the i-th AD. The above formulation belongs to a classic dynamic programming problem and can be solved with group Knapsack algorithm (Salkin and De Kluyver 1975). After obtaining the optimal $\{x_{ij}\}$ , optimal $\alpha$ can be retrieved accordingly.

Style comparison For an FMCG store, the launching of new clothing styles now and then is common practice. To reduce the marketing risk of awful designs, advertisers need to be aware of the market response to different styles quickly to adjust future production and inventory. Within CIA, different fashion styles should consume nearly equal impressions to compare their post-impression performance in a quantifiable way. Advertisers only need to put the n tested styles in the same campaign and let CIA take over. Given a comparable budget with keyword bidding, CIA solves the following optimization problem to allocate impressions to each style

$^{^{4}}$ For the ease of presentation, we assume that k is the same

among all ADs. In practice k could be varied due to bid settings of different ADs.

as uniform as possible.

$\max_{\alpha} - ||\mathbf{s} - \frac{\mathbf{1}^T \mathbf{s}}{n} \mathbf{1}||_2$ s.t. $(\beta - \epsilon) \mathbf{1}^T \tilde{\mathbf{z}} \le \mathbf{1}^T \mathbf{z} \le (\beta + \epsilon) \mathbf{1}^T \tilde{\mathbf{z}},$

$\beta \in (0, 1], \epsilon \in [0, \beta),$ (10)

where β and ǫ have the same meaning as in GMV optimization and the new variable s denotes the amount of impressions in all the ADs.

This problem can also be solved by group Knapsack algorithm following the same logic with GMV optimization. However, for the quadratic form of the objective, the optimal solution would yield uniform impressions among ADs. Based on this, we first calculate the feasible cost range as [b, B]. Then lⁱ and uⁱ are replayed to obtain a feasible range of αⁱ and also [bⁱ , Bⁱ ]. Following the notations, the style comparison problem can be reformulated as:

$\max_{\alpha} -||\mathbf{s} - \frac{\mathbf{1}^{T}\mathbf{s}}{n}\mathbf{1}||_{2}$

$s.t. \quad b_{i} \leq z_{i} \leq B_{i}, \forall i = 1, ..., n,$

$b \leq \mathbf{1}^{T}\mathbf{z} \leq B.$ (11)

The above formulation is a standard quadratic programming problem and can be solved with mature solvers such as OSQP (Stellato et al. 2017). After obtaining the optimal s, the replay module combined with the binary search method is utilized to get the optimal α. Compared with Knapsack algorithms, directly optimizing with quadratic programming provides a numeric solution without manual discretization.

4 Ranking with CIA

Within keyword bidding, there exists an obvious mismatch: platform tries to optimize ctr for revenue while advertisers care more about cvr. Meanwhile, consumer experience depending on conversions (consistent with advertisers) is also ignored. To mitigate the mismatch, industrial practice often introduces all kinds of relevant factors into the ranking formula (such as audience/advertiser experience) (Zhu et al. 2017) or implements a multi-stage ranking with different criteria (Liu et al. 2017) to balance advertisers’ profits. Nevertheless, all these strategies bring damage to the classic auction mechanism, affecting its interpretation and the platform’s revenue.

Given the new bidding formula, the ranking score in CIA has the following form:

$r = bid*ctr = \alpha*tk*cvr*ip*ctr = \alpha*tk*gmv. \eqno(12)$

Compared with the original ranking score, CIA takes explicitly cvr into bids while the ranking formula is preserved. In essence, budget savings from low-quality impressions are spent on higher-valued impressions, leading to a higher ROI. Therefore, the advertiser’s GMV increases together with platform revenue and audience experience.

We argue that the CIA bidding keeps the GSP auction mechanism unchanged, i.e., the allocation and charging mechanisms have remained the same. Advertisers can still change their keyword-level bid settings to influence tk so that the auction game still continues. For multi-agent behavior, the CIA bidding only takes individual tk and impression value into account, therefore bypasses the difficulty of modeling the competition’s uncertainty to a great extent. However, compared with keyword-level bidding, CIA bids on an impression-level and bases its decision on conversion value (which is typically unavailable for advertisers), thus helping alleviate advertisers’ irrationality and might shorten the time to reach Nash equilibrium in the repeated auction game. In this sense, the market is more efficient.

5 Replay based CIA implementation

To support optimization in the CIA framework, we need to predict the performance of different α on an AD. Particularly, we are interested in the following two functions:

f(α, AD, E) : daily cost of an AD if bidding based on α, g(α, AD, E) : daily GMV of an AD if bidding based on α, where E represents the external bidding environment. For the baseline cost, we have:

h(KB, AD, E) : daily cost of an AD with keyword bidding.

A historical auction log contains the contextual information of all the candidate ADs, bids, predicted ctr, cvr and commodity price etc. An asynchronous data collection system based on MaxCompute (Alibabacloud 2018) is designed to tackle the huge-volume streaming queries. For the replay-based prediction to be usable, the auction distribution of each AD should be relatively stationary. To justify the assumption, we chose four typical ADs to plot their daily auction statistics from Jan. 25, 2018 to Jan. 31, 2018 in Fig. 3. Furthermore, the corresponding cumulative CTR and CVR distributions are displayed in Fig. 4, which shows that CTR and CVR distributions are nearly the same for all the test ADs across the week. The two results together justify that the replay module is reliable enough to predict performance based on historical auction records.

Figure 3: Auction volumes of four ADs across a week, where the length of the rectangular displays the mean and the line shows the standard deviation (A-Home Fabric, B-Cosmetics, C-Female Clothing, D-Office Furniture).

With the historical daily auction log, the replay works as:

1. Updating the bids with α, and re-ranking all the ADs in the current auction according to eCPM;

Figure 4: CTR and CVR distributions of four ADs’ auctions across a week. The line represents the mean while the shadow represents the standard deviation.

• 2. For the top-N ADs 5 , calculating their cost c once clicked;
• 3. Accumulating ctr ∗ c and ctr ∗ cvr ∗ ip to the cost and GMV, respectively.

Keyword-level bid settings can also yield rough performance following the same procedure. Reversely, in the optimization process, a mapping from the AD’s daily cost to α would also be necessary. The inverse estimation of α can be solved efficiently via binary search due to monotonicity.

As a black box module, replay encapsulates the functions of winning probability/bid landscape estimation and bidding optimization together (see Fig. 2). Currently, it can generate a pair of (α, cost/GMV ) in milliseconds. To alleviate the influence of marketing shifts, CIA can average performance from the auction log of the past several days. Other parameterized representations can also fit the mappings (see (Ren et al. 2017) and references therein). However, we choose to replay for its time efficiency and validated accuracy in engineering practice.

6 Experimental Evaluation

In this section, we first introduce our experimental settings including baseline algorithms and evaluation metrics. Then we report some offline simulation and online A/B test results with further analysis and discussions.

6.1 Experimental settings

To test the performance of the CIA framework, we decompose evaluations into three levels, i.e., AD level, campaign level and platform level.

The AD level evaluation bids as equation (7) (abbreviated as CIA-AD) and is compared with keyword-level bids. The evaluation of CIA-AD is only used to illustrate the effectiveness of CIA over keyword-level bids. For the AD-level evaluation, we compare the performance of GMV, ROI, CVR, and PPC. Formally, CIA-AD is evaluated with the following baseline with approximately the same advertising cost (β = 1, ǫ = 0.1):

KB: Standard keyword-level bid settings are used to bid each impression.

Campaign level experiments constitute the core evaluations. We test the GMV optimization (CIA-CAMP1) and style comparison (CIA-CAMP2) performance in Section 3.2. The following baselines are used (β = 1, ǫ = 0.2):

CoKB: For campaign GMV optimization, joint optimization of budget allocation and keyword-level bids is implemented (Zhang et al. 2012), to compare with CIA-CAMP1.

KB-CAMP: For style comparison, advertisers set the same keyword bids on different ADs to compare their performance, which serves as the baseline of CIA-CAMP2. The normalized standard deviation of cost spent on different ADs in CIA-CAMP2 is compared.

The platform level experiments mainly focus on evaluating the effects on the platform when massive advertisers switch to CIA bidding. Specifically, advertiser GMV/ROI and platform revenue are evaluated simultaneously.

To make the results more persuasive, all the experiments are carried out on massive randomly selected ADs/campaigns across two weeks. For the sake of commercial privacy, all performance measures are presented in the form of a relative shifting ratio compared with the baseline.

6.2 Offline simulation

The offline simulation operates on historical auction logs and relies on replay for optimization and evaluation.

AD-level: We randomly choose 133, 743 ADs and test their performance with CIA-AD. The overall average performance and four examples are shown in Tab. 2

Table 2: Offline CIA-AD performance with KB as a baseline

Measures (%)	cost	GMV	ROI	CVR	PPC
Overall	−0.10	9.69	9.86	10.90	1.80
AD-A	0.90	6.00	5.10	6.20	1.00
AD-B	-0.09	9.54	9.64	6.03	4.66
AD-C	0.51	18.84	18.24	23.27	4.25
AD-D	0.53	6.74	6.18	6.55	0.35

From Tab. 2 we can see that with an approximately equal cost, the GMV and ROI of CIA-AD has an increase of nearly 10 percent. Meanwhile, CVR obtains the maximum increase of 10.9 percent, which means that more impressions with high conversion rates are acquired by CIA bidding. This coincides with our analysis as the bidding equation takes cvr

⁵N is the number of ad slots for a query.

into account. It can also be seen that with more cost, AD-A, C, and D even see an ROI elevation, which means that platform revenue also grows with advertisers’ ROIs.

Campaign-level: For a campaign-level offline simulation, the results of randomly selected 2599 campaigns which opt in CIA are shown in Tab. 3. It shows that with a nearly equal cost, CIA leads to an increase of GMV/ROI by 13 percent, which is reasonable since CoKB still bids on the keyword level although co-optimized. Meanwhile, CIA bids towards high-valued impressions in the most fine-grained way. The increase in CVR also validates the analysis.

Table 3: Offline CIA-CAMP1 performance with CoKB bidding as a baseline

Measures (%)	cost	GMV	ROI	CVR	PPC
Overall	1.18	14.64	13.62	17.10	3.80
CAMP-A	-0.25	15.70	15.99	28.41	11.22
CAMP-B	0.24	14.48	14.21	22.34	8.63
CAMP-C	-0.87	6.41	7.34	18.02	12.07
CAMP-D	2.25	25.00	22.25	32.45	8.35

For the style comparison demand, the performance of CIA-CAMP2 with 17334 campaigns is listed below in Tab. 4. Compared with naive equal keyword bids for style comparison, CIA-CAMP2 arrives a more uniform impression allocation, leading to a fair style comparison.

Table 4: Normalized standard deviations of offline style comparison performance

Strategies	CIA-CAMP2	KB-CAMP
Overall	0.660	0.790
CAMP-A	0.520	0.799
CAMP-B	0.852	0.951
CAMP-C	0.181	0.269
CAMP-D	1.562	1.687

Platform level: Multiple advertisers to opt in CIA will definitely change the global bidding environment, bringing complex consequences. To further evaluate the effects, we conduct an offline performance test by randomly switching a proportion of all the ADs to bid with CIA. The results of the whole platform (including all ADs) are displayed in in Tab. 5. It shows that with more and more advertisers changing to

Table 5: Offline platform performance with KB bidding as a baseline

Measures (%)	cost	GMV	ROI	CVR	PPC
10	0.25	0.58	0.33	0.72	0.43
30	0.58	3.42	2.82	2.31	1.00
50	0.76	4.79	4.00	3.99	1.33
100	1.41	9.08	7.5	8.18	1.63

CIA, the platform revenue (reflected by advertisers’ costs) and advertiser ROI grows together, which means that CIA has a positive effect on a healthy marketing ecosystem.

6.3 Online experiments

The online experiments compare different strategies by randomly allocating online queries to buckets implementing different strategies.

AD-level: We apply the replay module on the historical log to estimate a proper α to balance the cost. Then it is used online to evaluate the performance of CIA-AD as displayed in Tab.6, where”overall” summarizes the whole 213, 510 ADs and A,B,C,D are the same with those in offline evaluations. With a 2 percent increase in cost, overall

Table 6: Online CIA-AD performance with KB as a baseline

Measures (%)	cost	GMV	ROI	CVR	PPC
Overall	2.00	11.60	9.50	11.90	2.30
AD-A	-0.20	1.60	1.80	3.10	2.90
AD-B	8.80	12.80	19.10	2.07	-12.90
AD-C	7.90	25.30	16.10	41.40	1.70
AD-D	6.20	23.60	16.40	23.00	7.00

GMV and ROI encounters an increase of 11.6 and 9.5 percent, respectively. The PPC also sees an increase of 7.0 percent, which means that platform efficiency also increases.

Campaign-level: Online evaluation results of CIA-CAMP1 and CIA-CAMP2 with the same set of offline campaigns are listed in Tab. 7 and Tab .8, respectively. The fundamental increase of GMV, ROI, CVR and decrease of PPC follows the right logic. We can also see that in online experiments, different campaigns have various heterogeneous performances. This might be caused by the complex environmental shift due to bidding changes. Moreover, we conclude that offline simulation results are also accurate compared with the online version, which validates that our replay system is also reliable.

Table 7: Online CIA-CAMP1 performance with CoKB bidding as a baseline

Measures (%)	cost	GMV	ROI	CVR	PPC
Overall	0.01	12.50	11.50	3.70	−6.29
CAMP-A	3.18	28.67	24.68	17.47	4.66
CAMP-B	-1.58	43.99	46.32	4.35	-7.07
CAMP-C	-9.02	-10.61	-1.73	8.83	-1.71
CAMP-D	6.91	20.37	12.59	12.56	-4.73

Table 8: Normalized standard deviations of online style comparison performance

Strategies	CIA-CAMP2	KB-CAMP
Overall	0.524	0.574
CAMP-A	0.670	1.180
CAMP-B	0.621	0.970
CAMP-C	0.498	0.732
CAMP-D	0.092	0.587

Platform level: A large-scale online experiment with 25% ADs changing to CIA bidding is conducted. The overall ROI of the ADs with CIA bidding sees an 12.1 increase with a 4.2 increase in cost, showing that advertiser ROI and platform revenue grow together. For all the ADs, the average increase ratio is smaller due to the performance of non-CIA bidding ADs. In sum, although our derivation of CIA lies in the perspective of the advertiser, the result shows that CIAinduced ranking has a positive influence on both advertisers and platform.

Table 9: Online performance with KB as a baseline (25%) ADs switch to CIA-AD)

Measures (%)	cost	ROI	PPC	CTR
CIA ADs	4.2	12.1	1.8	1.4
All ADs	1.3	3.2	0.8	0.5

Conclusion 7

In this paper, we propose a unified advertiser demands optimization framework called CIA in the e-commerce sponsored search. CIA formulates formally various demands with budget and unit conversion cost constraints. To maintain the advertiser knob on ROI, CIA constructs an impression-level bidding equation with tk, which bridges the advertiser’s keyword-level bids and the accepted ROI. Furthermore, to optimize campaign-level demands, CIA introduces an additional cost regulator to adjust AD-level tkand derives the constraint of virtual budget from the historical log. We find that under the GSP mechanism, CIA balances the benefits of three parties in sponsored search together, leading to a healthy marketing ecosystem. We illustrate the CIA optimization framework with two most representative demands as campaign-level GMV optimization and style comparison. A replay system is implemented to solve the optimization with knapsack algorithm or quadratic programming. Offline and online experimental results show that CIA provides a useful bidding strategy and is also qualified as an optimization framework for various demands.

References

• [Alibabacloud 2018] Alibabacloud. 2018. Maxcompute. https://www.alibabacloud.com/help/product/27 #97.htm sales smash record with $25 billion haul. Accessed Jan. 28, 2018.
• [Cai et al. 2017] Cai, H.; Ren, K.; Zhang, W.; Malialis, K.; Wang, J.; Yu, Y.; and Guo, D. 2017. Real-time bidding by reinforcement learning in display advertising. In Proceedings of the Tenth ACM International Conference on Web Search and Data Mining, 661–670. ACM.
• [Claudia et al. 2012] Claudia, P.; Brian, D.; Rod, H.; Ori, S.; Troy, R.; and Foster, P. 2012. Bid optimization and inventory scoring in targeted online advertising. In The 18thACM SIGKDD Conference on Knowledge Discovery and Data Mining, 804-812. ACM.
• [Diemert et al. 2017] Diemert, E.; Meynet, J.; Lefortier, D.; and Galland, P. 2017. Attribution modeling increases efficiency of bidding in display advertising. arXiv preprint arXiv:1707.06409.
• [Gai et al. 2017] Gai, K.; Zhu, X.; Li, H.; Liu, K.; and Wang, Z. 2017. Learning piece-wise linear models from large scale data for ad click prediction. arXiv preprint arXiv:1704.05194.
• [Karande, Mehta, and Srikant 2013] Karande, C.; Mehta, A.; and Srikant, R. 2013. Optimizing budget constrained spend
• in search advertising. In Proceedings of the sixth ACM international conference on Web search and data mining, 697– 706. ACM.
• [King, Atkins, and Schwarz 2007] King, M.; Atkins, J.; and Schwarz, M. 2007. Internet advertising and the generalized second-price auction: Selling billions of dollars worth of keywords. The American economic review 97(1):242-
• [Krishna 2009] Krishna, V. 2009. Auction theory. Academic
• [Lee, Jalali, and Dasdan 2013] Lee, K. C.; Jalali, A.; and Dasdan, A. 2013. Real time bid optimization with smooth budget delivery in online advertising. In Proceedings of the Seventh International Workshop on Data Mining for Online Advertising, 1. ACM.
• [Liu et al. 2017] Liu, S.; Xiao, F.; Ou, W.; and Si, L. 2017. Cascade ranking for operational e-commerce search. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 1557-1565. ACM.
• [Qin, Chen, and Liu 2015] Qin, T.; Chen, W.; and Liu, T.-Y. 2015. Sponsored search auctions: Recent advances and future directions. ACM Transactions on Intelligent Systems and Technology (TIST) 5(4):60.
• [Ren et al. 2017] Ren, K.; Zhang, W.; Chang, K.; Rong, Y.; Yu, Y.; and Wang, J. 2017. Bidding machine: Learning to bid for directly optimizing profits in display advertising. IEEE Transactions on Knowledge and Data Engineering.
• [Reuters 2017] Reuters. 2017. http://fortune.com/2017/11/11/china-singles-day-a Accessed Jan. 28, 2018.
• [Salkin and De Kluyver 1975] Salkin, H. De Kluyver, C. A. 1975. The knapsack problem: a survey. Naval Research Logistics (NRL) 22(1):127-144.
• [Stellato et al. 2017] Stellato, B.; Banjac, G.; Goulart, P.; Bemporad, A.; and Boyd, S. 2017. OSQP: An operator splitting solver for quadratic programs. ArXiv e-prints.
• [Wang et al. 2017] Wang, Y.; Liu, J.; Liu, Y.; Hao, J.; He, Y.; Hu, J.; Yan, W.; and Li, M. 2017. Ladder: A humanlevel bidding agent for large-scale real-time online auctions. arXiv preprint arXiv:1708.05565.
• [Zhang et al. 2012] Zhang, W.; Zhang, Y.; Gao, B.; Yu, Y.; Yuan, X.; and Liu, T.-Y. 2012. Joint optimization of bid and budget allocation in sponsored search. In Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining, 1177–1185. ACM.
• Wang, J. 2014. Optimal real-time bidding for display advertising. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, 1077-1086. ACM.
• [Zhu et al. 2017] Zhu, H.; Jin, J.; Tan, C.; Pan, F.; Zeng, Y.; Li, H.; and Gai, K. 2017. Optimized cost per click in taobao display advertising. In Proceedings of the 23rd ACM