< List of probability distributions
A log linear distribution (or log linear model) is used to represent count data. As a discrete distribution, it can only assume a limited number of values. This model is an extension of the Poisson distribution and it sometimes called a Poisson regression model when used to model contingency tables.
The log linear distribution is a right-skewed distribution defined by its mean and variance. The mean corresponds to the logarithm of the expected count values, while the variance is the square of the logarithm of the expected count values.
Log linear models are frequently used in statistical analyses, including contingency table analysis and chi-squared testing. Additionally, it is applied in machine learning domains such as natural language processing and text mining.
Uses for the log linear distribution
Commonly used in statistical analyses such as contingency table analysis and chi-squared testing, loglinear models are also applied in machine learning fields like natural language processing and text mining:
- Contingency table analysis: Log linear models can be applied to examine contingency tables, which display the frequency of co-occurring categorical variables. For instance, a contingency table may present the frequency of various cancer types within different age groups. Log-linear models allow
for tests of homogenous associations in I × J × K and higher-dimensional tables [1]. - Chi-squared testing: Loglinear models can be used during chi-squared testing, which are statistical tests comparing observed frequencies with expected frequencies. For example, a chi-squared test might assess the observed frequency of distinct cancer types against the expected frequency based on age groups.
- Natural language processing: Log linear models can be used to analyze natural language data, such as text and code. For instance, they can be used to evaluate the frequency of specific words in a document or the frequency of distinct code constructs within a program.
- Text mining: Log linear models can be harnessed for extracting patterns and trends from text data. For example, they can identify commonly co-occurring words or frequently used code constructs.
Poisson regression vs. log linear
Poisson regression is only used for numerical, continuous data. But the same technique can model categorical explanatory variables or counts in the contingency table cells. When used in this way, the models are called log linear models. Log-linear models determine the relationship between cell counts and the levels of categorical variables. They are designed to represent association and interaction patterns among categorical variables.
Log-linear modeling is well-suited for Poisson, Multinomial, and Product-Multinomial sampling. These models are particularly suitable when there isn’t a distinct separation between the response and explanatory variables or when multiple responses are present.
The two-way log-linear model can be defined as follows: Let µij be expected counts, E(nij), in an I × J table. An model analogous to two-way ANOVA is [2]
with the following constraints, which compensate for overparameterization:
Other names for the log linear distribution
Log-linear models haven many different names, depending on what field you’re working in [3]. For example:
- Multinomial Logistic Regression is also called polytomous, polychotomous, multi-class logistic regression, or simply multilogit regression.
- Maximum Entropy Classifier: Since logistic regression estimation adheres to the maximum entropy principle, it is occasionally termed “maximum entropy modeling,” with the resulting classifier called the “maximum entropy classifier.” This term was more common in natural language processing (NLP) literature in the 1990s; a more popular term today is the following neural network classification.
- Neural Network – Single Neuron Classification: Binary logistic regression is equivalent to a single-layer, single-output neural network with a logistic activation function trained under log loss. This is sometimes called classification using a single neuron.
- Generalized Linear Model and Softmax Regression: Logistic regression is a generalized linear model featuring the logit link function. The logistic link function is occasionally referred to as softmax and, due to its use of exponentiation for converting linear predictors into probabilities, it is sometimes called an exponential model.
References
[1] https://personal.psu.edu/abs12/stat504/Lecture/lec16.pdf
[2] Introduction to log-linear models. Retrieved June 9, 2023 from: https://personal.psu.edu/abs12/stat504/Lecture/lec16.pdf
[3] O’Connor, B. (2014). Lecture 8 Multiclass/Log-linear models, Evaluation, and Human Labels.