Home
Articles
How Natural Language Processing can...

How Natural Language Processing can Revolutionize Human Resources

Written by Raja Sengupta

8 minutes read

Natural language processing is an ever-growing interest area in the analytics application spectrum and is relevant to HR. In fact, it can revolutionize the quality of insights. In this article, we will explain you how.

Natural language processing has a significant relevance to HR

Did you know that text analysis has been the most prevalent productivity tool over the past 3 decades or so for HR? It is very familiar to HR.

HR has been using Boolean keyword searches for identifying good resumes/ job applications for a long time already. However, often with unpredictable and humorous results.

Natural language processing (NLP) takes text analysis to the much higher level of detail, granularity, and accuracy. Acute insights from NLP were a technological constraint in the past but there have been major strides of late. This has aided by the development of distributed computing and from the intense research in NLP applications by academic and professional bodies around the world.

Related: Predictive Analytics in HR

The essence of people function lies in an effective analysis of communication and natural language is the most prevalent medium of human communication. However, the scope of NLP in people function needs to be spearheading by operational HR alone.

Most HR business engagement generates high volumes of natural language, which is unstructured data. Think about areas like recruitment, employee feedback, surveys, appraisals, learning, legal cases, counseling etc.

Additionally, legacy HR processes and forms can be re-engineered to accumulate ever increasing volumes of natural language data. Via an active policy of audio recording & transcribing or even a slight redesign of various HR processes forms/surveys/applications.

Key benefits for HR with reference to natural language processing

Benefits are many, corresponding to varying levels of engagement and investment by HR.

It starts from generic text analytics (sentiment analysis). Goes to advanced insights (via computational linguistics models) and can even include potential semi-automation.

Once implemented, such services can be delivering via APIs and database connectivity. Or even from standalone client based systems. Google and Microsoft are prime big player examples in the API NLP space.

How do the insights from natural language processing analysis impact HR?

HR specific NLP analysis, with varying and often progressive levels of insights not only acts as decision supports (DSS). But also, enable greater accuracy and speed to key HR business processes and improving HR metrics. They also reduce human bias in decision-making application. Examples include resume scoring and survey analysis.

Often NLP systems act as “first or second level filtering” or “hypothesis proofs” to corroborate human decisions in HR

Related: Do you know all 21 Employee Performance Metrics?

The NLP evolution curves below indicates the needs of text analytics and computational linguistics as it maximizes the business benefits of NLP.

Structured and unstructured data synergize to improve the quality of insights for HR.

For instance, key traditional areas for HR data modeling have been attrition, absenteeism, career paths, compensation & benefits, etc. For such models, the insights gained through NLP can fit in as explanatory variable thereby improving the accuracy of the model.

Usually, HR processes forms like employee survey, feedback, assessments have consisted of several structured data points (check, radio, drop down, slide boxes, etc.). The NLP can be used to further incorporate and capitalize the open answers in this survey. This improves the quality of insights.

Taken together, they both improve metrics of HR processes.

Misinformation with regards to adoption of natural language processing in HR Processes

It is not the case that natural language processing systems replace HR. On the contrary, these systems further empower HR personnel within their organization.

The complexities of human language, communication and dynamic decision making required by HR in the real world is complex. This implies that total automation is impractical and can be downright counterproductive. Machines find it complex to comprehend the finer nuances of human language. Like sarcasm, ambivalence, deformed compliments, passive aggression, regional norms, etc.

An interesting and somewhat parallel comparison is in the case of autopilots and flies by wire systems. They have been around for two decades or more. But never replace humans in cockpits, although in simulation tests they outperform human pilots.

Drones (theoretically pilot-less) are also controlled by a human pilot.

The concept of full automation is completely misplaced. Job losses are actually skill restructuring/ retraining/ realignment program. Not result from automation and may be progressively required.

Bottlenecks in adopting natural language processing to HR

There aren’t many vendors who are only focused on advanced NLP to HR processes yet. Most vendors are text analytics generalists; they may not have in-depth aware of HR specific challenges. OrganizationView is a good example of a dedicated operator in this space and there are a few more.

Other key bottlenecks are HR data security/protection, data accessibility, quality, API integration. The engagement and collaboration programs between HR and IT also have scope for improvement in this area.

Related: A list of 14 Human Resource Metrics

Large strides have been made in recent times about the application of NLP to other areas. For instance, NLP enables service providers to process vast amounts of data and make predictions on bankruptcy evaluations and contracts in the legal sector. Script writing is revolutionized using NLP in the entertainment business and now the time is ripe for adoption in HR.

Identified approaches in NLP that are relevant to HR

Operational HR should take the lead and identity relevant application areas within their own organizations. The impact of NLP in HR is likely to depend upon data availability, security, integration, company policy or any other specific business requirements.

Broadly there are three aspects to applying NLP to HR.

1. Types of Generic natural language processing insights (relevant to the HR application context)

Sentiment Analysis of HR documents
Deep Information Extraction from HR documents
Classification/ ranking of HR documents as per business specifications
Automated Summation of HR documents (topic discovery)
Establishing HR Hypothesis and process improvement (a part of prescriptive analytics)

2. Application areas of natural language processing (within the HR application context)

Application/ Resume classification and scoring
Appraisal and 360-degree feedback analysis
Surveys and feedback analysis
Identifying Training, Succession planning
Social media content analysis of employees
Insights on documented Legal cases/ suits
Design and insights about Employee Counseling
NLP on virtually any unstructured data within the scope of HR, including transcribed data.

3. Overview of various NLP methodologies employed by vendors (within the HR application context)

Statistical Tagging

Statistical tagging offers insights from various levels of granularity starting from basic text classification, sentiment analysis to deep information extraction and topic modeling/ automated summation. Some of the popular information extraction/ topic discovery approaches are Conditional Random Fields, Hidden Markov Models, and LDA.

Symbolic Tagging

The HR familiarity with basic Boolean keyword searches to identify good resumes is a very good example of symbolic tagging. But today NLP models like nested, iterative and conditional “regular expressions” can fine tune symbolic tag searches to the deepest possible levels of granularity.

A combination approach of statistical and symbolic tagging is often referred to as a “conditional rules model” within the NLP context. Tailored combinations of “conditional rules models” are typically developed via integrated cohort analysis in collaboration with HR.

Related: A list of HR Analytics Training Courses

This may also help to establish evidence-based HR Hypothesis. And effectively push forward major HR initiatives to the organizational leadership.

A business case of NLP in a key HR process (Hiring)

The basic approach of natural language processing remains more or less the same across all types of unstructured data. However, for the sake of familiarity let’s take the example of resume scoring in Hiring on a large unstructured dataset

Here NLP can help on resume classification, ranking, deep extraction, identification and semi-automation in the hiring process

Classify and rank resumes according to their core skills, experience or any other priory. Like desirable skills and professional experience.
Classify resumes according to their format styles. Like chronological, reverse chronological, hybrid, skills-based, and qualification based functional based formats.
Identifying basic sections of a resume (topic model based on the priority given by HR)
Identify gaps in professional/ academic records in resumes
Identify potential fraud/ incorrect information and anomalies in resumes
Deep information extraction from resumes. For instance combination of professional skills/ education + university rankings + professional experiences + environment and context + international assignments/ location specific + awards/ recognitions + recommendations/ professional network ) via compound “conditional rules models”

Apart from resume/ application scoring, “Conditional rules models” can also help identify complex human language expressions. Like sarcasm, ambivalence, deformed compliments, passive aggression, this might be important for HR surveys, feedbacks, forums, social media data etc.

However, the degrees of accuracy may differ (and it’s an ongoing research area)

In systems where applications/ resumes have semi-structured data points (for example applications received through an online XML form), NLP can act in conjunction with the structured data points (SQL) for improving the quality and accuracy of classifications and inferences.

Once developed, semi-automation can also be applied to NLP models to enable

Periodic and automated evaluation of dataset via batch jobs and database procedures/triggers/functions
Automated scoring and classification of datasets via above
Sending an automated email to shortlisted candidates (for example a test set or interview call) or sending consolidated or specific reports to the HR/Recruitment team.

All these could reduce cost for the recruiter and add more accuracy in candidate screening. The NLP approach can definitely find the better candidates for a job application without any human biases.

Typical services offered by NLP vendors

NLP vendors typically offer a combination of services mentioned above, including summation, topic modeling, and conditional rules models.

However, don’t be fooled. Innovative marketing and promotion schemes might give the impression that there is a variety of computational approaches in NLP among different vendors – even though they are fundamentally the same.

It is therefore important for operational HR to have a good overview and to be able to discern NLP applications relevant to their specific business requirements and constraints

Related: What is HR Analytics?

NLP application service delivery might be through API services/ database integration or standalone implementations on clients (windows based installations). Static periodic reporting system (for example process improvement via a six sigma framework) would add advantage here.

The Future…

HR is the prime candidate for adoption of NLP-based technologies, as HR is inherently people-centric and communication based. HR business processes thus generate vast amounts of natural language data.

This presents an opportunity for HR. It will also enable HR to have greater intelligence and leverage within the organization.

Early adopters and pioneers don’t hesitate to contact us for further engagement!

Written by Raja Sengupta & Soumyasanto Sen
Soumyasanto Sen is a professionally Consultant/Manager/Advisor/Investor in HR Tech. He has 12+ years of experiences focusing on Strategies, Analytics, Cloud, UX, Security, Integration and Entrepreneurship in Digital HR Transformation.

Subscribe to our weekly newsletter to stay up-to-date with the latest HR news, trends, and resources.

Weekly update

Stay up-to-date with the latest news, trends, and resources in HR

Raja Sengupta

Raja Sengupta has approximately 20 years of experience in Analytics consulting with almost 10 years spent in people analytics consulting for clients in Europe. He has worked with key people analytics consultants and HR tech startups as a consultant and in a direct senior management role for fortune 500 conglomerates in the Europe region. Since 2021, he has diversified in finance analytics and geospatial analytics along HR analytics, synergizing analytics learning from all three disciplines.