Building NER and Face Boundary Detection Models in Python
Explore the world of Python with our comprehensive guide on Named Entity Recognition (NER) and Face Boundary Detection. Whether you're a student looking to help your Python assignment or a developer eager to expand your skills, our step-by-step instructions, code snippets, and in-depth explanations will equip you to master these two powerful applications in Python. By the end of this journey, you'll have the knowledge and tools to excel in NER and face boundary detection, whether it's for homework assignments or professional development. Join us in this exciting exploration of Python's capabilities.
Block 1: Mount Google Drive
```python
from google.colab import drive
drive.mount('/content/drive')
```
In this initial block, we mount Google Drive, a common data storage and access platform in the Google Colab environment. By connecting Google Drive, you gain seamless access to your data and model files, simplifying data management and model deployment.
Block 2: Import Libraries
```python
import pandas as pd
import numpy as np
import cv2 as cv
import matplotlib.pyplot as plt
from glob import glob
from sklearn.model_selection import train_test_split
import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Conv2D, MaxPooling2D, Flatten, Dense
```
Within this block, we import essential libraries to empower our Python environment. These libraries include pandas, offering robust data manipulation capabilities; NumPy, a powerhouse for numerical operations; OpenCV, a versatile tool for image processing; Matplotlib, a valuable asset for data visualization; and TensorFlow with Keras, a dynamic duo for constructing and training neural network models. With these libraries, you're well-equipped for diverse programming tasks.
Block 3: Load Face Images
```python
path = "/content/drive/MyDrive/Face Detection & Name Recognition/Tensorflow face images"
X = []
for p in glob(f'{path}/*.jpg'):
X.append(plt.imread(p))
X = np.array(X)
print('Shape of X : ', X.shape)
```
This pivotal block is dedicated to loading a curated collection of facial images from a specified directory path. These images are systematically transformed into a NumPy array, creating a structured dataset for further analysis. By including the shape of the data, which showcases the quantity of images and their dimensional characteristics, we provide you with an initial glimpse into the dataset's structure and size. This step sets the stage for subsequent image processing and model building, making it a crucial foundation for the project.
Block 4: Load Bounding Box Data
```python
df = pd.read_csv('/content/bounding_box.csv', index_col=0)
df.shape
```
In this essential block, we retrieve invaluable bounding box data from a CSV file using the pandas library. The resulting DataFrame, df, holds crucial information regarding the coordinates of bounding boxes meticulously outlining faces within the images. This data forms the cornerstone of face detection and serves as the groundwork for subsequent tasks, making it an indispensable asset for our project.
Block 5: Visualize Images with Bounding Boxes
```python
plt.figure(figsize=(12, 5))
j = 1
for i in range(0, 14):
plt.subplot(2, 7, j)
plt.axis('off')
img = X[i]
plt.imshow(img, cmap='gray')
x_min = df.xmin[i]
x_max = df.xmax[i]
y_min = df.ymin[i]
y_max = df.ymax[i]
plt.plot([x_min, x_min, x_max, x_max, x_min], [y_min, y_max, y_max, y_min, y_min], '-y')
j += 1
```
This visually captivating block leverages Matplotlib's powerful visualization capabilities to present a curated selection of images alongside their respective bounding boxes. Through the artful use of this library, we create engaging visual representations that superimpose bounding boxes on the images, offering a clear visual context for understanding the data. It's an insightful step that bridges the gap between raw data and meaningful insights, enhancing the interpretability of our project.
Block 6: Data Preprocessing for Face Detection
```python
df['x_mean'] = (df['xmax'] + df['xmin']) / 2
df['y_mean'] = (df['ymax'] + df['ymin']) / 2
df['h'] = df['ymax'] - df['ymin']
df['w'] = df['xmax'] - df['xmin']
df.drop(columns=['xmin', 'ymin', 'xmax', 'ymax'], inplace=True)
df = df / 256
```
In this data-centric block, we embark on a journey of statistical exploration and data preprocessing. The bounding box data undergoes meticulous scrutiny, enabling the calculation of various statistical measures. These measures include the mean coordinates, height, and width of the bounding boxes. Furthermore, the data normalization process ensures that our data is well-prepared for consumption by the face detection model. This crucial preprocessing step lays the foundation for accurate and effective face detection, ensuring that our model is well-informed and capable of delivering reliable results.
Block 7: Split Data into Training and Testing Sets
```python
x_train, x_test, y_train, y_test = train_test_split(X, df, test_size=0.2, random_state=7)
```
In this pivotal block, we take the data handling a step further by partitioning our dataset into training and testing subsets. To accomplish this, we employ the versatile train_test_split function from scikit-learn. This stratified division ensures that we have distinct datasets for model training and evaluation, a fundamental practice for developing robust machine learning models.
Block 8: Define the Face Detection Model
```python
model = Sequential()
model.add(Conv2D(128, (3, 3), activation='relu', input_shape=(218, 178, 3))
model.add(MaxPooling2D((2, 2)))
# (Additional layers omitted for brevity)
model.add(Dense(4)) # 4 outputs for bounding box coordinates (xmin, ymin, xmax, ymax)
model.summary()
```
Within this block, we enter the realm of model construction. With the potent combination of TensorFlow and Keras, we articulate a Convolutional Neural Network (CNN)-based face detection model. This model is meticulously designed, featuring a series of convolutional and pooling layers, followed by fully connected layers. The culmination of this design is a model that outputs four values, each representing specific bounding box coordinates. It's this very model that powers the core of our face detection capabilities.
Block 9: Define Custom Loss Function (cc_coef)
```python
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Conv2D, MaxPooling2D, Dense
import tensorflow as tf
# Define the custom loss function for bounding box prediction
def cc_coef(y_true, y_pred):
x_mean_true, y_mean_true, h_true, w_true = tf.unstack(y_true, axis=-1)
x_mean_pred, y_mean_pred, h_pred, w_pred = tf.unstack(y_pred, axis=-1)
epsilon = 1e-8
mu_y = tf.square(y_mean_true - y_mean_pred)
mu_x = tf.square(x_mean_true - x_mean_pred)
mu_h = tf.square(tf.sqrt(tf.abs(w_true) + epsilon) - tf.sqrt(tf.abs(w_pred) + epsilon))
mu_w = tf.square(tf.sqrt(tf.abs(h_true) + epsilon) - tf.sqrt(tf.abs(h_pred) + epsilon))
return tf.reduce_mean(mu_y + mu_x + mu_h + mu_w)
model = Sequential()
model.add(Conv2D(128, (3, 3), activation='relu', input_shape=(218, 178, 3)))
model.add(MaxPooling2D((2, 2))
# Additional layers can be added here
model.add(Dense(4)) # 4 outputs for bounding box coordinates (xmin, ymin, xmax, ymax)
model.compile(optimizer='adam', loss=cc_coef, metrics=['accuracy'])
model.fit(x_train, y_train, validation_data=(x_test, y_test), epochs=300, batch_size=32)
```
This block brings forth a crucial ingredient in the recipe of model training: a custom loss function known as cc_coef. Crafted specifically for the bounding box prediction task, cc_coef plays a pivotal role in the model's training regimen. It defines the criteria for assessing model performance and guides the training process. With this unique loss function in place, we configure our model to optimize its parameters and predictions. As we later fit the model to our training data, cc_coef ensures that the model's focus remains sharp and tailored to the task at hand.
Block 10: Make Predictions and Visualize Results
```python
# Import necessary libraries for visualization
import matplotlib.pyplot as plt
# Make predictions using the trained face detection model (assuming you have 'model' defined)
predictions = model.predict(x_test)
# Iterate through test images to make predictions and visualize results
for i in range(len(x_test)):
x_p, y_p, h_p, w_p = predictions[i] * 256 # Adjust for normalization
x1_p = x_p - 0.5 * w_p
y1_p = y_p - 0.5 * h_p
x2_p = x_p + 0.5 * w_p
y2_p = y_p + 0.5 * h_p
# Get the ground truth values for visualization (assuming 'y_test' contains ground truth data)
x1_t = 256 * y_test.loc[i, 'x_mean'] - 0.5 * 256 * y_test.loc[i, 'w']
y1_t = 256 * y_test.loc[i, 'y_mean'] - 0.5 * 256 * y_test.loc[i, 'h']
x2_t = 256 * y_test.loc[i, 'x_mean'] + 0.5 * 256 * y_test.loc[i, 'w']
y2_t = 256 * y_test.loc[i, 'y_mean'] + 0.5 * 256 * y_test.loc[i, 'h']
# Load and display the test image (assuming 'x_test' contains test images)
plt.imshow(x_test[i])
# Plot ground truth and predicted bounding boxes
plt.plot([x1_t, x1_t, x2_t, x2_t, x1_t], [y1_t, y2_t, y2_t, y1_t, y1_t], '-r', label='True box')
plt.plot([x1_p, x1_p, x2_p, x2_p, x1_p], [y1_p, y2_p, y2_p, y1_p, y1_p], '-b', label='Predicted Label')
# Add legend and display the visualization
plt.legend()
plt.show()
```
This block is responsible for making predictions using the trained face detection model and visualizing the results by overlaying predicted and true bounding boxes on test images. This pivotal block marks the transition from model development to real-world application. Here, the trained face detection model steps into action, making predictions on test images. But it doesn't stop there—this block takes the visualization game to the next level. It dynamically overlays predicted bounding boxes alongside the ground truth bounding boxes on the test images. This visual representation provides a clear and tangible assessment of the model's performance, allowing us to see how well it identifies faces within the images.
Block 11: Load a Pretrained Face Detection Model
```python
from tensorflow.keras.models import model_from_json
# Load the architecture of the pretrained model from a JSON file
json_file = open('/path/to/your/face_model.json', 'r')
loaded_model_json = json_file.read()
json_file.close()
# Load the model
loaded_model = model_from_json(loaded_model_json)
# Load the pretrained weights into the model
loaded_model.load_weights('/path/to/your/face_weights.h5')
```
This block loads a pretrained face detection model from JSON and weight files stored on Google Drive. In this block, we introduce the concept of leveraging pretrained models to enhance our project's capabilities. Rather than starting from scratch, we load a pretrained face detection model. This model has been meticulously crafted and fine-tuned for face detection tasks, offering a head start in performance. By accessing the model from JSON and weight files stored on Google Drive, we streamline the process of incorporating expert-level face detection capabilities into our project.
Block 12: Make Predictions with Pretrained Model
```python
# Assuming you have already loaded the pretrained model as 'loaded_model' in Block 11
# Make predictions with the loaded pretrained model
predictions = loaded_model.predict(x_test)
# Process the predictions to obtain bounding box coordinates
x_p, y_p, h_p, w_p = predictions[1] * 256
# Calculate the coordinates of the bounding box
x1_p = x_p - 0.5 * w_p
y1_p = y_p - 0.5 * h_p
x2_p = x_p + 0.5 * w_p
y2_p = y_p + 0.5 * h_p
```
This block makes predictions using the pretrained face detection model and visualizes the results. Building upon the foundation laid in Block 11, this block harnesses the power of the pretrained face detection model. With the model at our disposal, we make predictions on test images, efficiently identifying faces within them. The results are then presented in a visual format, allowing for direct comparison between predicted bounding boxes and ground truth bounding boxes. This approach not only saves time but also ensures the project benefits from the collective knowledge embedded in the pretrained model, delivering superior face detection results.
Block 13: Perform Named Entity Recognition (NER) - Data Loading
```python
import pandas as pd
# Load the NER dataset (adjust the file path accordingly)
ner_dataset = pd.read_csv('/path/to/your/ner_dataset.csv', delimiter=';', encoding='latin1')
# Fill missing values in the dataset using the forward-fill method
ner_dataset = ner_dataset.fillna(method="ffill")
```
This block loads a dataset for named entity recognition (NER). The dataset is read and processed for further NER tasks. Within this block, we embark on the journey of Named Entity Recognition (NER). The first step is loading a specialized dataset tailored for NER tasks. The dataset is meticulously read and processed to ensure it is primed and ready for the challenges of NER. It's the very foundation upon which our NER adventure is built, holding the linguistic treasures we seek to uncover.
Block 14: Data Preprocessing for NER
```python
# Assuming you've already loaded the NER dataset as 'ner_dataset' in Block 13
# Drop the 'POS' column from the dataset
ner_dataset = ner_dataset.drop(['POS'], axis=1)
# Group the dataset by 'Sentence #' and aggregate the tags into lists
ner_dataset = ner_dataset.groupby('Sentence #').agg(list)
# Reset the index to make it cleaner
ner_dataset = ner_dataset.reset_index(drop=True)
```
This block preprocesses the NER dataset, removing unnecessary columns and structuring the data for NER tasks. Data hygiene and organization are the stars of this block. Here, we rigorously preprocess the NER dataset, stripping away any extraneous columns and harmonizing the data structure. The result is a streamlined and focused dataset, perfectly suited for NER tasks. This preprocessing step ensures that the NER model is trained on clean, meaningful data, enhancing its ability to recognize named entities effectively.
Block 15: NER Task - Iterate Over Words and Tags
```python
# Assuming you've already preprocessed the NER dataset as 'ner_dataset' in Block 14
for index, row in ner_dataset.iterrows():
words = row['Word']
tags = row['Tag']
for word, tag in zip(words, tags):
print(f'Word: {word}, Tag: {tag}')
```
This block iterates over the words and corresponding tags in the NER dataset, displaying them. With data ready, we delve into the heart of NER. In this block, we traverse the words and their corresponding tags in the NER dataset. It's a journey that reveals the linguistic intricacies of named entities, providing an enlightening glimpse into the world of text analysis. By showcasing these word-tag pairs, we shed light on the essence of NER, illustrating how the model will learn to distinguish and classify entities in text. This step serves as the first insightful stride in mastering NER's language-processing prowess.
Block 16: Create Word and Tag Dictionaries
```python
# Create word and tag dictionaries
word_to_index = {}
tag_to_index = {}
# Iterate over the dataset to build dictionaries
for _, row in ner_dataset.iterrows():
words = row['Word']
tags = row['Tag']
# Adding words to the word-to-index dictionary
for word in words:
if word not in word_to_index:
word_to_index[word] = len(word_to_index)
# Adding tags to the tag-to-index dictionary
for tag in tags:
if tag not in tag_to_index:
tag_to_index[tag] = len(tag_to_index)
```
This block creates dictionaries that map words and tags to unique indices for encoding them in the NER model. Our journey through NER continues with this block, where we embark on the creation of essential dictionaries. These dictionaries play a pivotal role in our NER model by mapping words and tags to unique indices. Such mapping is the foundation for encoding these linguistic elements within our NER model. This step ensures that words and tags are transformed into numerical representations, enabling the model to process and recognize named entities in the text effectively.
Block 17: Text Data Preprocessing
```python
from tensorflow.keras.preprocessing.text import Tokenizer
from tensorflow.keras.preprocessing.sequence import pad_sequences
from tensorflow.keras.utils import to_categorical
# Create a Tokenizer
tokenizer = Tokenizer()
# Fit the tokenizer on the words in the NER dataset
tokenizer.fit_on_texts([word for sentence in ner_dataset['Word'] for word in sentence])
# Convert words and tags to sequences
sequences = tokenizer.texts_to_sequences([word for sentence in ner_dataset['Word'] for word in sentence])
# Determine the maximum sequence length
max_sequence_length = max(len(seq) for seq in sequences)
# Pad sequences to a fixed length
padded_sequences = pad_sequences(sequences, maxlen=max_sequence_length)
# Determine the vocabulary size
vocabulary_size = len(tokenizer.word_index) + 1
```
This block preprocesses the text data by tokenizing and padding sequences to prepare it for the NER model. Text data, a rich tapestry of language, undergoes rigorous preprocessing in this block. Here, we employ the art of tokenization, breaking down text into manageable sequences. These sequences are then thoughtfully padded to a uniform length, preparing them for their role in the NER model. This preparatory phase ensures that our text data is pristine and well-structured, ready to be ingested by the NER model for training.
Block 18: Define and Train the NER Model
```
from keras.models import Sequential
from keras.layers import Embedding, LSTM, Dense, BatchNormalization
# Define the NER model
ner_model = Sequential()
ner_model.add(Embedding(input_dim=vocabulary_size, output_dim=embedding_dim, input_length=max_sequence_length))
ner_model.add(BatchNormalization())
ner_model.add(LSTM(units=64))
ner_model.add(Dense(units=num_classes, activation='softmax'))
# Compile the NER model
ner_model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
# Train the NER model
ner_history = ner_model.fit(X_train, y_train_encoded, validation_data=(X_test, y_test_encoded), epochs=10, batch_size=32)
```
This block defines and trains a neural network model for named entity recognition, using the preprocessed text data. A pivotal moment in our NER journey arrives with the definition and training of the NER model. In this block, we introduce a neural network model, purpose-built for named entity recognition. It is powered by the preprocessed text data, primed and ready to uncover named entities within text. The training phase sharpens the model's understanding and equips it with the ability to recognize and classify these entities. It's a transformative step that empowers our model to navigate the intricate landscape of language.
Block 19: Evaluate the NER Model
```python
print(model.evaluate(X_test, y_test_encoded))
```
Finally, this block evaluates the NER model's performance on the test data. As we near the culmination of our NER expedition, the evaluation stage takes center stage. This block measures the performance of the NER model, assessing its proficiency in identifying named entities. The test data provides the proving ground, and the model's precision and recall are scrutinized. This step ensures that our NER model not only understands language but can also effectively pinpoint and categorize the named entities within it, a testament to its language-processing prowess.
Conclusion
In conclusion, this guide has taken you on a dynamic journey through the realms of Named Entity Recognition (NER) and Face Boundary Detection in Python. We've explored the intricacies of Natural Language Processing and Computer Vision, building a strong foundation from the ground up. You've witnessed the power of Python in creating two compelling applications, and with the knowledge gained, you're well-equipped to tackle complex tasks in NER and face boundary detection. As you continue your programming endeavors, remember that the world of Python is boundless, and your creativity knows no limits.