Market Segmentation using Conjoint Analysis
Conjoint Series — Scene 5
Conjoint Series — Scene 1 — How to Understand Consumer Preference?
Conjoint Series — Scene 2 — Which Product are Most Important?
Conjoint Series — Scene 3 — Designing Orthogonal Survey
Conjoint Series — Scene 4 — Regression Analysis for All Respondent
“Hi, Ray, Abe and Ran. Are you ready to listen to my lecture on how to design market segment using the conjoint analysis?” Dess ask and smile.
“Sir… yes sir…” Abe and Ray nod enthusiastically while Ran giving her sweetest smile today.
When we create the segment on previous discussion, we put number of segment three as the example. However, how do we know that three segments is enough? Why not four segments or five segments? This has been a quite discussion and many methods emerge to calculate the optimum segment, such as using elbow method, using silhouette coefficient, or gap statistics [1]. Some recent method also emerge, such as using BIC to evaluate optimum number of cluster[2].
In our discussion today, we will use the most commonly used to determine the number of cluster, the elbow method. Please be aware that most common does not necessarily best method. We use elbow method to give the intuitive guidance on creating optimum segment and finding the segment characteristics.
“Lets start with segmentation today using three segments and observe the output when we run the code. Abe, please do the magic.” Dess instruct Abe to start running the code.
caSegmentation(y=tpref, x=tprof, c=3)In elbow method, our objective is to minimize the error within clusters. However, when we run this, we will get so many output. Our focus is to extract the value of within segment error. We can ignore the many outputs and only care where we can extract information required.
Available components:
[1] "cluster" "centers" "totss" "withinss" "tot.withinss" "betweenss" "size"
[8] "iter" "ifault"The total error within segment is captured in component number 5, tot.withinss.
“Now please, create a loop to run segmentation from 2 to 10 clusters and capture the error within segment.”
# initiating data frame
num_cluster <- c(seq(2,10))
ss_in <- rep(0,9)
ss_within=data.frame(
num_cluster,
ss_in
)
for (i in 1:9){
clu = caSegmentation(y=tpref,x=tprof,c=i+1)
ss_within$ss_in[i]=clu$segm[5]
}
# change data type
options(digits = 9)
ss_within$num_cluster = as.factor(ss_within$num_cluster)
ss_within$ss_in = as.numeric(ss_within$ss_in)To decide the optimal number of clusters, we should plot the total error within segment. Using observation and judgement, we can decide optimal cluster is where change of error is not significant.
library(ggplot2)
p <- ggplot(ss_within, aes(x = num_cluster, y = ss_in)) +
geom_bar(stat = "identity",fill="blue")+
ggtitle("Total Error Whitin Cluster x Number of Cluster")+
xlab("Number of Cluster")+
ylab("Total Error within Cluster")
p
After 6 clusters, it seems that the error doesn’t drop significantly. So, lets decide that the number of clusters is 6. Please note that this is not right or wrong decision. We can decide the optimal number of cluster based on what is making sense for the business.
Now, the question is, does six segment is quite good? Can we plot if those segment are clean and to intersection between member of segments?
We can plot our segment in two dimensional space. But, since in our case we have 11 level of tea characteristics and plotting 11 dimensional space is hard, we will use other statistical tools called PCA (Principal Component Analysis) and take the two highest dimension where the result explain majority of the variation in the data.
# optimum number of clusters is 6
clu = caSegmentation(y=tpref,x=tprof,c=6)
# preparation
prtutil_tea = caPartUtilities(y=tprefm, x=tprof, z=tlevn)
prtutil_tea = as.data.frame(prtutil_tea)
# running pca
res.pca <- prcomp(prtutil_tea, scale = TRUE)
print(res.pca)
summary(res.pca)
# plotting PCA dim 1 and dim 2 with segment
library(factoextra)
fviz_pca_ind(res.pca,
geom.ind = "point", # show points only (nbut not "text")
col.ind = as.factor(clu$sclu), # color by groups
addEllipses = TRUE, # Concentration ellipses
ellipse.type = "convex",
legend.title = "Groups"
)
The result is not well align. Segment 3 for example have huge intersection with segment 2. Two kind of decision we can make when we are facing this conditions:
- Accept this condition — considering that Dim1(26.6%) + Dim2(22.6%) only explain 49.9% of the variation of the data, we can conclude that there might be other information that can explain and differentiate segment 3.
- Reject this condition — combine segment 3 with segment 2.
We can now observe the segment characteristics.
“What kind of characteristics that we want over here?” Abe ask Dess.
“Lets start with what we have from the output from Conjoint function. Extract the importance factors and the utility”
“All right, let me do it” Abe start writing the script. It is not as easy as he thought at first time because the output from Conjoint is not save in list like other function in R. He use function to save the verbose (output), extract the value from there and write it in Excel.
# --------------------Segment Profile --------------
# profile segment 1
tprefm1 <- tprefm[clu$sclu==1,]
tpref1 <- data.frame(Y=matrix(t(tprefm1), ncol=1, nrow=ncol(tprefm1)*nrow(tprefm1), byrow=F))
Conjoint(y=tpref1, x=tprof, z=tlevn)
verbose_output_1 <- capture.output({
Conjoint(y=tpref1, x=tprof, z=tlevn)
})
# extracting utility
util_1 <- as.data.frame(verbose_output_1[28:39])
colnames(util_1)[1] = "utility"
util_1[c('row', 'level_util')] <- str_split_fixed(util_1$utility,' ',2)
util_1$level_util <- trimws(util_1$level_util,which='both')
util_1[c('level','util_seg1')] <- str_split_fixed(util_1$level_util,' ',2)
util_1 <- util_1 %>% select(level,util_seg1)
# extracting factor importance
importance_1<-as.data.frame(verbose_output_1[41])
colnames(importance_1)[1]="importance"
importance_1[c('row','val')]=str_split_fixed(importance_1$importance,' ',2)
importance_1[c('price','val')]=str_split_fixed(importance_1$val,' ',2)
importance_1[c('variety','val')]=str_split_fixed(importance_1$val,' ',2)
importance_1[c('kind','val')]=str_split_fixed(importance_1$val,' ',2)
importance_1[c('aroma','val')]=str_split_fixed(importance_1$val,' ',2)
importance_1 = importance_1 %>% select(price,variety,kind,aroma)
# continue for segment 2 to segment 6
# --------------
# combining result utility
util_all = util_1 %>% left_join(util_2,by="level") %>%
left_join(util_3,by="level") %>%
left_join(util_4,by="level") %>%
left_join(util_5,by="level") %>%
left_join(util_6,by="level")
# combining result importance
importance_all = rbind(importance_1,importance_2,importance_3,importance_4,importance_5,importance_6)
# segment characteristic using Excel
library(writexl)
write_xlsx(util_all,"util_all.xlsx")
write_xlsx(importance_all,"importance_all.xlsx")Abe edit the output in Excel to be more presentable and easy to digest. Here is the result.
“What will be your opinion looking at this result Ran?” Dess pinpointing Ran, the smart business analyst.
“Well, first of all, we cannot combine Segment 3 and Segment 2 because they have difference importance factor in deciding which tea the customers wanted to purchase. Segment 3, the most important factor is variety….” Ran exhale her breath in excitement, and continue “while segment 2, the decision factor is the kind of tea”.
“Good observation. Anything else?” asked Dess
“Hmmm… it looks like that Segment 4 and Segment 1 is quite similar in term of importance. What I mean is, the customers on those segment really do not have any peculiarity on the decision to buy tea. Their importance is almost the same for each variable.”
“Again Ran, that is also good observation. But before we decide if we can combine segment 4 and segment 1, let see what the detail look like. Abe, please show the excel for each of the level.”
“Owh… I see now,” Ran murmuring, “Segment 1 and Segment 4 differ in the variety that the customer prefer. Segment 1 prefer green tea because it has high and positive utility while Segment 4 prefer black tea. Segment 1 also sensitive to price while Segment 4 is quite agnostic”
“This is actually fantastic result. Once we can summarized our finding to our client, I think they would love it because they understanding the customers better.” Ran continue thinking and also giving her opinion.
“Dess, once we have the segment, what can we do?” she asked.
“There is one more thing that conjoint analysis can do to us. Creating market simulation based on the result and the segment available. Lets continue discussion next week.”
Note:
The R script is long and not clean. To see overall script, please see in here
Source:
[1] Determining The Optimal Number Of Clusters: 3 Must Know Methods — Datanovia
[2] http://cs.uef.fi/sipu/pub/BIC-acivs2008
[4] https://www.geeksforgeeks.org/how-to-split-column-into-multiple-columns-in-r-dataframe/
