More organizations are adopting data-driven and technology-focused approaches to business and hence the need for analytics expertise continues to grow. As a result, career opportunities in analytics are around every corner. Due to this identifying analytics talent has become a priority for companies in nearly every industry, from healthcare, finance, and telecommunications to retail, energy, and sports.
In this blog, we will be talking about different career paths and option in the Business Analytics field. Furthermore, we will be discussing the qualifications required for being a business analyst and what are the primary roles a Business Analyst handles at a firm
In this blog, we will be discussing
Business analysts, also known as management analysts, work for all kinds of businesses, nonprofit organizations, and government agencies. Certainly, job functions can vary depending on the position, the work of business analysts involves studying business processes and operating procedures in search of ways to improve an organization’s operational efficiency and achieve better performance. Simply put, a Business Analyst is someone who works with people within an organization to understand their business problems and needs and then to interpret, translate and document those business needs in terms of specific business requirements for solution providers to implement.
Most entry-level business analyst positions require at least a bachelor’s degree. Therefore beginning Business Analysts need to have either a strong business background or extensive IT knowledge. Likewise, you can start to work as a business analyst with job responsibilities that include collecting, analyzing, communicating and documenting requirements, user-testing and so on. Entry-level jobs may include industry/domain expert, developer, and/or quality assurance.
With sufficient experience and good performance, a young professional can move into a junior business analyst position. In contrast, some choose instead to return to school to get master’s degrees before beginning work as business analysts in large organizations or consultancies.
Professional business analysts play a critical role in a company’s productivity, efficiency, and profitability. Hence, essential skill sets range from communication and interpersonal skills to problem-solving and critical thinking. Let us discuss each in a bit more detail
First of all, Business analysts spend a significant amount of time interacting with clients, users, management, and developers. Therefore, being an effective communicator is key. You will be expected to facilitate work meetings, ask the right questions, and actively listen to your colleagues to take in new information and build relationships.
Every project you work on is, at its core, is around developing a solution to a problem. Business analysts work to build a shared understanding of problems, outline the parameters of the project, and determine potential solutions. Hence, problem-solving skill is a must-have for this job position.
A business analyst is an intermediary between a variety of people with various types of personalities: clients, developers, users, management, and IT. Therefore, you have to be able to achieve a profitable outcome for your company while finding a solution for the client that makes them happy. This balancing act demands the ability to influence a mutual solution and maintaining professional relationships.
Business analysts must assess multiple choices before leading the team toward a solution. Effectively doing so requires a critical review of data, documentation, user input surveys, and workflow. They ask probing questions until every issue is evaluated in its entirety to determine the best conflict resolution. Therefore, critical thinking skill is a must have pre-requisite for this job position.
A career path of a business analyst usually begins with working at an entry level, and gradually with experience and with acquiring a better understanding of how businesses function, growing up the ladder.
Also, Business Analysts enjoy a seamless transition to different roles according to one’s interest because the profession consists of a set of skills which are highly specialized and can be applied to any industry and to any subject matter area successfully. As a result, this allows for the Business Analyst to move between industry, company and subject matter area with ease which becomes their career progression and a focus of professional development.
Other roles that one can take up after gaining experience as a Business Analyst can be
Once you have several years of experience in the industry, you will finally reach a pivotal turning point where you can choose the next step in your business analyst career. After three to five years, you can be positioned to move up into roles such as IT business analyst, senior/lead business analyst or product manager.
But broadly beyond all the fancy names given to designations, we can consider four levels of professional analytics roles:
Level 1: The Business Analyst
Level 2: The Data Scientist
Level 3: The Analytics Decision Maker
Level 4: The Analytics Leader
Modern Analyst identifies several characteristics that make up the role of a business analyst as follows:
The average salary of a business analyst in India is around 6.5 L.P.A. As one continues to gain the experience in this field, the salary gets more lucrative.
The more experience you have as a business analyst, the more likely you are to be assigned larger and/or more complex projects. After eight to 10 years in various business analysis positions, you can advance to chief technology officer or work as a consultant. You can take the business analyst career path as far as you would like, progressing through management levels as far as your expertise, talents, and desires take you.
So with so many interesting, promising and rewarding options available for Business Analysts, they need to first get a firm hold about the basics of data analysis. You can also have a look at this post to know more about what are the different components in data science. It will help you to boost your business analyst career.
We, at Dimensionless Technologies, offer data science course which helps to make you industry ready. Do go through our website and let us know how we may help you.
We provide online courses on Data Science and Business Analytics.Sign up today to learn data science and business analytics online.
Sourced through Scoop.it from: dimensionless.in
Bi-Variate Analysis
by Samridhi Dutta | Jan 5, 2017
In my previous post, we have covered Uni-Variate Analysis as an initial stage of data-exploration process. In this post, we will cover the bi-variate analysis. Objective of the bi-variate analysis is to understand the relationship between each pair of variables using statistics and visualizations. We need to analyze relationship between:
the target variable and each predictor variable two predictor variables
It is important to analyze the relationship between the predictor and the target variables to understand the trend for the following reasons:
The bi-variate analysis and our model should communicate the same story. This will help in understand and analysing the accuracy of our models, and to make sure, that our model has not over-fit the training data. If the data has too many predictor variables, we should include only those predictor variables in our regression models which show some trend with the target variable. Our aim with the regression models is to understand the story each significant variable is communicating, and its behaviour with other predictor variables and the target variable. A variable that has no pattern with the target variable, may not have a direct relation with the target variable (while a transformation of this variable might have a direct relation). If we understand the correlations and trends between the predictor and the target variables, we can arrive at better and faster transformations of predictor variables, to get more accurate models faster.
Eg. Following curve indicates logarithmic relation between target and predictor variables. A curve of below-mentioned shape indicates that there is a logarithmic relation between x and y. In order to transform the above curve into linear, we need to take an exponential of 10 of the predictor variable. Hence, a simple scatter plot can give us the best estimate of variables – transformations required to arrive at the appropriate model.
It is important to understand the correlations between each pair of predictor variables. Correlated variables lead to multi-collinearity. Essentially, two correlated variables are transmitting the same information, and hence are redundant. Multi-collinearity leads to inflated error term and wider confidence interval (reflecting greater uncertainty in the estimate). When there are too many variables in a data-set, we use techniques like PCA for dimensionality reduction. Dimensionality reduction techniques work upon reducing the correlated variables, to reduce the extraneous information and so that we run our modelling algorithms on the variables that explain the maximum variance.
We have understood why bi-variate analysis is an essential step to data exploration. Now we will discuss the techniques to bi-variate analysis.
For illustration, we will use Hitters data-set from library ISLR. This is Major League Baseball Data from the 1986 and 1987 seasons. It is a data frame with 322 observations of major league players on 20 variables. The target variable is Salary while the rest of the 19 are dependent variables. Through this data-set, we will demonstrate bi-variate analysis between:
two continuous variables, one continuous and one categorical variable, two categorical variables
data(Hitters) hitters=Hitters summary(hitters) ## AtBat Hits HmRun Runs ## Min. : 16.0 Min. : 1 Min. : 0.00 Min. : 0.00 ## 1st Qu.:255.2 1st Qu.: 64 1st Qu.: 4.00 1st Qu.: 30.25 ## Median :379.5 Median : 96 Median : 8.00 Median : 48.00 ## Mean :380.9 Mean :101 Mean :10.77 Mean : 50.91 ## 3rd Qu.:512.0 3rd Qu.:137 3rd Qu.:16.00 3rd Qu.: 69.00 ## Max. :687.0 Max. :238 Max. :40.00 Max. :130.00 ## ## RBI Walks Years CAtBat ## Min. : 0.00 Min. : 0.00 Min. : 1.000 Min. : 19.0 ## 1st Qu.: 28.00 1st Qu.: 22.00 1st Qu.: 4.000 1st Qu.: 816.8 ## Median : 44.00 Median : 35.00 Median : 6.000 Median : 1928.0 ## Mean : 48.03 Mean : 38.74 Mean : 7.444 Mean : 2648.7 ## 3rd Qu.: 64.75 3rd Qu.: 53.00 3rd Qu.:11.000 3rd Qu.: 3924.2 ## Max. :121.00 Max. :105.00 Max. :24.000 Max. :14053.0 ## ## CHits CHmRun CRuns CRBI ## Min. : 4.0 Min. : 0.00 Min. : 1.0 Min. : 0.00 ## 1st Qu.: 209.0 1st Qu.: 14.00 1st Qu.: 100.2 1st Qu.: 88.75 ## Median : 508.0 Median : 37.50 Median : 247.0 Median : 220.50 ## Mean : 717.6 Mean : 69.49 Mean : 358.8 Mean : 330.12 ## 3rd Qu.:1059.2 3rd Qu.: 90.00 3rd Qu.: 526.2 3rd Qu.: 426.25 ## Max. :4256.0 Max. :548.00 Max. :2165.0 Max. :1659.00 ## ## CWalks League Division PutOuts Assists ## Min. : 0.00 A:175 E:157 Min. : 0.0 Min. : 0.0 ## 1st Qu.: 67.25 N:147 W:165 1st Qu.: 109.2 1st Qu.: 7.0 ## Median : 170.50 Median : 212.0 Median : 39.5 ## Mean : 260.24 Mean : 288.9 Mean :106.9 ## 3rd Qu.: 339.25 3rd Qu.: 325.0 3rd Qu.:166.0 ## Max. :1566.00 Max. :1378.0 Max. :492.0 ## ## Errors Salary NewLeague ## Min. : 0.00 Min. : 67.5 A:176 ## 1st Qu.: 3.00 1st Qu.: 190.0 N:146 ## Median : 6.00 Median : 425.0 ## Mean : 8.04 Mean : 535.9 ## 3rd Qu.:11.00 3rd Qu.: 750.0 ## Max. :32.00 Max. :2460.0 ## NA’s :59
#deleting rows with missing values in target variable hitters=hitters[-which(is.na(hitters$Salary)),] #seperating continuous and factor variables hitters_factor=data.frame(1:nrow(hitters)) hitters_cont=data.frame(1:nrow(hitters)) i=1 while(i<=length(names(hitters))) { if(class(hitters[,i])==”factor”) hitters_factor=cbind(hitters_factor,hitters[i]) else hitters_cont = cbind(hitters_cont,hitters[i]) i=i+1 } hitters_cont = hitters_cont[,-1] hitters_factor=hitters_factor[,-1]
Please note, that the way to do the bi-variate analysis is same irrespective of predictor or target variable.
To do the bi-variate analysis between two continuous variables, we have to look at scatter plot between the two variables. The pattern of the scatter plot indicates the relationship between the two variables. As long as there is a pattern between two variables, there can be a transformation applied to the predictor / target variable to achieve a linear relationship for modelling purpose. If no pattern is observed, this implies no relationship possible between the two variables. The strength of the linear relationship between two continuous variables can be quantified using Pearson Correlation.A correlation coefficient of -1 indicates high negative correlation, 0 indicates no correlation, 1 indicates high positive correlation.
Correlation is simply the normalized co-variance with the standard deviation of both the factors. This is done to ensure we get a number between +1 and -1. Co-variance is very difficult to compare as it depends on the units of the two variables. So, we prefer to use correlation for the same.
Please note: * If two variables are linearly related, it means they should have high Pearson Correlation Coefficient. * If two variables are correlated does not indicate they are linearly related. This is because correlation is deeply impacted by outliers. Eg. the correlation for both the below graphs is same, but the linear relation is not there in the second graph:
If two variables are related, it does not mean they are necessarily correlated. Eg. in the below graph of y=x^2-1, x and y are related but not correlated (with correlation coefficient of 0). x=c(-1, -.75, -.5, -.25, 0, .25, .5, .75, 1) y=x^2-1 plot(x,y, col=”dark green”,”l”)
cor(x,y) ## [1] 0
cor(hitters_cont[1:4]) ## AtBat Hits HmRun Runs ## AtBat 1.0000000 0.9639691 0.5551022 0.8998291 ## Hits 0.9639691 1.0000000 0.5306274 0.9106301 ## HmRun 0.5551022 0.5306274 1.0000000 0.6310759 ## Runs 0.8998291 0.9106301 0.6310759 1.0000000
This gives the correlation-coefficients between the continuous variables in hitters data-set. Since it is difficult to analyse so many values, we prefer to attain a quick visualization of correlation through scatter plots. Following command gives the scatter plots for the first 4 continuous variables in hitters data-set:
pairs(hitters_cont[1:4], col=”brown”)
Linear pattern can be observed between AtBat and Hits, and can be confirmed from the correlation value = 0.96 Linear pattern can be observed between Hits and Runs, and can be confirmed from the correlation value = 0.91 Linear pattern can be observed between AtBat and Runs, and can be confirmed from the correlation value = .899
plot(hitters_cont[,1], hitters_cont[,2], col=”yellow”, xlab=”AtBat”, ylab=”Hits”)
The graph shows a strong positive linear correlation.
cor(hitters_cont[,1], hitters_cont[,2]) ## [1] 0.9639691
Correlation value of .96 verifies our claim of strong positive correlation. We can obtain better visualizations of the correlations through library corrgram and corrplot:
library(corrgram) corrgram(hitters)
Strong blue means strong +ve correlation. Strong red means strong negative correlation. Dark color means strong correlation. Weak color means weak correlation.
library(corrplot) continuous_correlation=cor(hitters_cont) corrplot(continuous_correlation, method= “circle”, type = “full”, is.corr=T, diag=T)
This gives a good visual representation of the correlation and relationship between variables, especially when the number of variables is high. Dark blue and large circle represents high +ve correlation. Dark red and large circle represents high -ve correlation. Weak colors and smaller circles represent weak correlation.
2-way Frequency Table: We can make a 2-way frequency table to understand the relationship between two categorical variables.
head(hitters_factor) ## League Division NewLeague ## -Alan Ashby N W N ## -Alvin Davis A W A ## -Andre Dawson N E N ## -Andres Galarraga N E N ## -Alfredo Griffin A W A ## -Al Newman N E A tab=table(League=hitters_factor$League, Division=hitters_factor$Division)#gives the frequency count tab ## Division ## League E W ## A 68 71 ## N 61 63
This gives the frequency count of League vs Division
Chi-Square Tests of Association: To understand if there is an association / relation between 2 categorical variables.
chisq.test(tab) ## ## Pearson’s Chi-squared test with Yates’ continuity correction ## ## data: tab ## X-squared = 0, df = 1, p-value = 1
The chisquare probability value = 1, we retain the null hypothesis that there is not sufficient association between the two factor variables.
Visualization of two categorical variables can be obtained through Heat Map and Fluctuation Charts as illustrated in the example:
library(ggplot2) runningcounts.df <- as.data.frame(tab) ggplot(runningcounts.df, aes(League,Division)) + geom_tile(aes(fill = Freq), colour = “black”) + scale_fill_gradient(low = “white”, high = “red”)+theme_classic()
Dark Color represents higher frequency and lighter colors represent lower frequencies.
ggplot(runningcounts.df, aes(League, Division)) + geom_point(aes(size = Freq, color = Freq, stat = “identity”, position = “identity”), shape = 15) + scale_size_continuous(range = c(3,15)) + scale_color_gradient(low = “white”, high = “black”)+theme_bw()
Dark Color represents higher frequency and lighter colors represent lower frequencies.
Aggregations can be obtained using functions xtabs, aggregate or using dplyr library. Eg. In the Hitters Data-set, we will use Factor Variable: “Division”” and Continuous Var: “Salary”.
xtabs(hitters$Salary ~ hitters$Division) #gives the division-wise sum of salaries ## hitters$Division ## E W ## 80531.01 60417.50 aggregate(hitters$Salary, by=list(hitters$Division), mean,na.rm=T) #gives the division-wise mean of salaries ## Group.1 x ## 1 E 624.2714 ## 2 W 450.8769 hitters%>%group_by(Division)%>%summarise(Sum_Salary=sum(Salary, na.rm=T), Mean_Salary=mean(Salary,na.rm=T), Min_Salary=min(Salary, na.rm=T),Max_Salary = max(Salary, na.rm=T)) ## # A tibble: 2 × 5 ## Division Sum_Salary Mean_Salary Min_Salary Max_Salary ## <fctr> <dbl> <dbl> <dbl> <dbl> ## 1 E 80531.01 624.2714 67.5 2460 ## 2 W 60417.50 450.8769 68.0 1900 T-Test: 2-Sample test (paired or unpaired) can be used to understand if there is a relationship between a continuous and a categorical variable. 2-sample T test can be used for categorical variables with only two levels. For more than two levels, we will use Anova.
Eg. Conduct T-test on Hitters data-set to check if Division (a predictor factor variable) has an impact on Salary (continuous target variable). H0: Mean of salary for Divisions E and W is same, so there is not much impact for divisions on salary HA: Mean of salay is different; so there is a deep impact of divisions on salary
df=data.frame(hitters$Division, hitters$Salary) head(df) ## hitters.Division hitters.Salary ## 1 W 475.0 ## 2 W 480.0 ## 3 E 500.0 ## 4 E 91.5 ## 5 W 750.0 ## 6 E 70.0 library(dplyr) df%>%filter(hitters.Division ==”W”)%>%data.frame()->sal_distribution_W df%>%filter(hitters.Division==”E”)%>%data.frame()->sal_distribution_E x=sal_distribution_E$hitters.Salary y=sal_distribution_W$hitters.Salary t.test(x, y, alternative=”two.sided”, mu=0) ## ## Welch Two Sample t-test ## ## data: x and y ## t = 3.145, df = 218.46, p-value = 0.001892 ## alternative hypothesis: true difference in means is not equal to 0 ## 95 percent confidence interval: ## 64.73206 282.05692 ## sample estimates: ## mean of x mean of y ## 624.2714 450.8769
2-sided indicates two tailed test. P-value of .19% indicates that we can reject the null hypothesis and there is a significant difference in mean salary based on division. Hence, division does make an impact on the salary.
Anova: Analysis of variance (ANOVA) is a collection of statistical models used to analyze the differences among group means and their associated procedures (such as “variation” among and between groups).
Anova
anova=aov(Salary~Division, data = hitters) summary(anova) ## Df Sum Sq Mean Sq F value Pr(>F) ## Division 1 1976102 1976102 10.04 0.00171 ** ## Residuals 261 51343011 196717 ## — ## Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ‘ 1
Since probability value is < 1%, so, the difference between the average salary for different divisions is significant.
Visualization can be obtained through box-plots, bar-plots and density plots.
Box Plots to establish relationship between Division variable and Salary Variable in Hitters data-set
x=plot(hitters$Division, hitters$Salary, col=”red”, xlab=”Division”, ylab=”Salary”)
x ## $stats ## [,1] [,2] ## [1,] 67.500 68 ## [2,] 215.000 165 ## [3,] 517.143 375 ## [4,] 850.000 725 ## [5,] 1800.000 1500 ## ## $n ## [1] 129 134 ## ## $conf ## [,1] [,2] ## [1,] 428.8074 298.5649 ## [2,] 605.4786 451.4351 ## ## $out ## [1] 1975.000 1861.460 2460.000 1925.571 2412.500 2127.333 1940.000 1900.000 ## ## $group ## [1] 1 1 1 1 1 1 1 2 ## ## $names ## [1] “E” “W”
The output indicates that there are 129 observations for E and 134 observations for W.
The statistics of the output indicate:
Minimum salary values for E and W divisions are: 67.5, 68 respectively.
Salary values at 1st quartile for E and W divisions are 215 and 165 respectively.
Salary values at 2nd quartile for E and w divisions are 517.14 and 375 respectively. This implies the median salary value of E division is 50% higher than W.
Salary values at 3rd quartile for E and W divisions are 850 and 725 respectively.
Maximum salary values for E and W divisions are 1800 and 1500 respectively.
Outliar values are indicated by $out values as classified in the $group.
Outliar salary values are 1975, 1861.46, 2460, 1925.57, 2412.50, 2127.33, 1940 for Division E (group 1).
Outliar salary value is 1900 for division W (group 2).
We can also use ggplot to make more beautiful visuals:
p=ggplot(hitters,aes(x=Division, y=Salary)) p+geom_boxplot(aes(color=Division, fill=Division))+theme_classic()+xlab(“Division”)+ylab(“Salary”)
To understand the distribution of first 4 continuous variable with respect to division:
plot(hitters_cont[1:4], col=hitters$Division)
plot(hitters$Salary, col=hitters$Division, lwd=2, cex = as.numeric(hitters$Division), ylab=”Salary”) legend(230, 2500, c(“E”, “W”), pch=1, col=(1:2))
The plot shows lower salaries for W while higher frequency of high salaries for E.
hitters%>%group_by(Division)%>%summarize(Salary=sum(Salary))%>%data.frame()->df df$Percentage=round(df$Salary/sum(df$Salary),3) df ## Division Salary Percentage ## 1 E 80531.01 0.571 ## 2 W 60417.50 0.429 p=ggplot(df, aes(x=Division, y=Salary, fill=Division)) p+geom_bar(stat=”identity”, alpha=.7)+ geom_text(aes(label=Percentage, col=Division, vjust=-.25))+guides(color=FALSE)+ theme_classic()+ggtitle(“Division-Wise Salary”)
The above graph gives a visualization of frequency count of Salary buckets as per the division
p=ggplot(hitters, aes(x=Salary)) p+geom_histogram(aes(fill=Division), position = “dodge”, bins=30)+xlab(“Salary”)+theme_classic()
p=ggplot(hitters, aes(x=Salary)) p+geom_histogram(aes(fill=Division), position = “stack”, alpha=.7)+xlab(“Salary”)+theme_classic() ## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
p+geom_histogram(aes(color=Division, fill=Division), bins=25, alpha=.3)+facet_grid(Division~.)+xlab(“Salary”) + theme_classic()
The histogram gives a clear demarcated pictorial representation of the salary for the 2 divisions.
p=ggplot(hitters, aes(x=Salary, col=Division, fill=Division)) p+geom_density(alpha=.4)+theme_classic() + ylab(“Density”)
The plot indicates high density for division W in the salary range 0 to 1000.
Bi-Variate Analysis provides a visual representation of the inter-relationship between the predictor variables. If the correlation between the predictor variables is high, then we would have to reduce the correlated variables in order to avoid multi-collinearity in the prediction models.
We also need to understand the bi-variate relationship between target variables and the predictor variables based on which we understand, analyze and validate our modeling results.
In essence it is one of the most important steps which gives us the insights on the interaction between the variables.
Read more at Dimensionless.in
