A service-oriented approach to the black box problem

The black box problem occurs in recommender systems when the recommender rationale is opaque towards the end user, causing decreased levels of confidence in the system. If the recommender system is considered as a service, this phenomenon can be described in terms of a value leak where there is asymmetry in the service bundles.

In this article we try to describe this problem from a service-based perspective and present a possible solution to this problem accordingly. The solution presented here is to partially solve asymmetry through a so-called white box model.



Recommender systems try to filter out suggestions from an item space based a user’s behaviour. By modeling preference through past inferences with the data set, the recommender system can predict the user’s affinity towards an item. Recommender systems have become popular in particular for online retail as a response to the long tail phenomenon [8]. They enable to “connect supply and demand, introducing consumers to these new and newly available goods and driving demand down the tail” [1].

Based on a definition proposed by Mercado et al. [6], services are a “meaningful bundle of technologies resources and assets”. The meaning of a service is typically captured in the Service Level Agreement (SLA) and represents the value of the service. There are various ways in which service value can be achieved. An important remark is that both service provider and consumer contribute to the value of a service. Co-creation of value in Web 2.0 and Enterprise 2.0 occurs through two-way interactions in which both actors strive for a global optimum. Mercado et al. argue that from this perspective businesses are to carefully consider issues that deviate from these global optima [6]. In this respect, a recommender system can be seen as a service that addresses long tail effects. Value is created through the retrieval of interesting products for the client, and for the provider through the sale of these products.

Imperfect service bundles

The co-creation of value can be challenged, causing value erosion or value leaks. Mercado et al. [6] identify several causes for value leaks. They list three scenarios in which service imperfections may arise: incompleteness of service bundles,
asymmetry in either service bundles themselves, or in the meaning for service bundles. Incompleteness occurs when certain technologies, resources and assets are missing. In contrast, asymmetry refers to situation in which technology, resources and assets are in fact present, but not equally distributed among the relevant parties [6].

Recommender systems may suffer from the black box problem. This problem emerges when the end user cannot form an adequate mental model of the recommender rationale, causing the user to be reluctant to accept recommendations[5]. This illustrates asymmetry in information between the service provider and consumer, introducing a value leak. The retailer loses profits as the customer no longer buys suggested products, and the client no longer enjoys interesting items or information.

Solving asymmetry

Two general approaches exist to deal with asymmetry in service bundles: either asymmetry is solved, or it is maintained. The choice of the applied strategy is context-dependend. In the case of recommender systems, it might not be desired to disclose too many details on the recommendation algorithm [5]. Significant research may have been dedicated to the development of these algorithms, and as a result, revealing the complete recommender rationale may introduce a new value leak [6].

To solve the black box problem an explanation system can be used to provide insight into the recommendation process. For example Herlocker et al. [5] explored a so called white box model for collaborative filtering. Collaborative filtering is a recommender strategy in which the system looks for similar users to the active user, i.e., neighbours. Items owned by one user, but not by the second, can be recommended to this second user. The predicted affinity for this item will depend on the similarity score between these profiles [8]. In their paper, Herlocker et al. compare this to a word-of-mouth recommendation style, and list three significant steps in the recommendation process [5]:

  1. User enters profile of ratings;
  2. ACF system locates people with similar profiles;
  3. Neighbours’ ratings are combined to form recommendations.

From these steps, they derived a number of means for explanation that were later implemented for a movie recommender system and were tested through user studies. One of their conclusions was that the explanation systems indeed increased acceptance of suggestions significantly[5], and as a result creating value and closing this value leak to some extent.

Other explanation systems have been developed since, such as PeerChooser, Pharos, SFVis, SmallWorlds, and TasteWeights [7, 9, 3, 4, 2]. The systems listed here all use visualizations and most of them use interaction as well to allow the user to alter some of the parameters used in the recommendation process for further refinement and insight gaining.


[1] C. Anderson. The Long Tail: Why the Future of Business Is Selling Less of More. Hyperion, 2006.

[2] S. Bostandjiev, J. O’Donovan, and T. Höllerer. Tasteweights: a visual interactive hybrid recommender system. In Proceedings of the sixth ACM conference on Recommender systems, RecSys ’12, pages 35{42, New York, NY, USA, 2012. ACM.

[3] L. Gou, F. You, J. Guo, L. Wu, and X. L. Zhang. Sfviz: interest-based friends exploration and recommendation in social networks. In Proceedings of the 2011 Visual Information Communication – International Symposium, VINCI ’11, pages 15:1-15:10, New York, NY, USA, 2011. ACM.

[4] B. Gretarsson, S. Bost, C. Hall, and T. Höllerer. Smallworlds: Visualizing social recommendations. Eurographics/ IEEE-VGTC Symposium on Visualization 2010, 2010.

[5] J. L. Herlocker, J. A. Konstan, and J. Riedl. Explaining collaborative filtering recommendations. In Proceedings of the 2000 ACM conference on
Computer supported cooperative work, CSCW ’00, pages 241-250, New York, NY, USA, 2000. ACM.

[6] C. Mercado, G. Dedene, E. Peters, and R. Maes. Toward a dynamic, systemic, and holistic theory for strategic value creation in ict-based services. A Focused Issue on Competence Perspectives on New Industry Dynamics (Research in Competence-Based Management, Volume 6), pages 153-207, 2012.

[7] J. O’Donovan, B. Smyth, B. Gretarsson, S. Bostandjiev, and T. Hollerer. Peerchooser: visual interactive recommendation. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’08, pages 1085-1088, New York, NY, USA, 2008. ACM.

[8] A. Rajaraman, J. Leskovec, and J. Ullman. Mining of Massive Datasets. Cambridge University Press, 2012.

[9] S. Zhao, M. X. Zhou, Q. Yuan, X. Zhang, W. Zheng, and R. Fu. Who is talking about what: social map-based recommendation for contentcentric social websites. In Proceedings of the fourth ACM conference on Recommender systems, RecSys ’10, pages 143-150, New York, NY, USA, 2010. ACM.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s