3. Packages and applications

This part of the Scientific Python Lectures is dedicated to various scientific packages useful for extended needs.

• 3.1. Statistics in Python
  • 3.1.1. Data representation and interaction
    • 3.1.1.1. Data as a table
    • 3.1.1.2. The pandas data-frame
  • 3.1.2. Hypothesis testing: comparing two groups
    • 3.1.2.1. Student’s t-test: the simplest statistical test
    • 3.1.2.2. Paired tests: repeated measurements on the same individuals
  • 3.1.3. Linear models, multiple factors, and analysis of variance
    • 3.1.3.1. “formulas” to specify statistical models in Python
    • 3.1.3.2. Multiple Regression: including multiple factors
    • 3.1.3.3. Post-hoc hypothesis testing: analysis of variance (ANOVA)
  • 3.1.4. More visualization: seaborn for statistical exploration
    • 3.1.4.1. Pairplot: scatter matrices
    • 3.1.4.2. lmplot: plotting a univariate regression
  • 3.1.5. Testing for interactions
  • 3.1.6. Full code for the figures
  • 3.1.7. Solutions to this chapter’s exercises
• 3.2. Sympy : Symbolic Mathematics in Python
  • 3.2.1. First Steps with SymPy
    • 3.2.1.1. Using SymPy as a calculator
    • 3.2.1.2. Symbols
  • 3.2.2. Algebraic manipulations
    • 3.2.2.1. Expand
    • 3.2.2.2. Simplify
  • 3.2.3. Calculus
    • 3.2.3.1. Limits
    • 3.2.3.2. Differentiation
    • 3.2.3.3. Series expansion
    • 3.2.3.4. Integration
  • 3.2.4. Equation solving
  • 3.2.5. Linear Algebra
    • 3.2.5.1. Matrices
    • 3.2.5.2. Differential Equations
• 3.3. scikit-image: image processing
  • 3.3.1. Introduction and concepts
    • 3.3.1.1. scikit-image and the scientific Python ecosystem
    • 3.3.1.2. What is included in scikit-image
  • 3.3.2. Importing
  • 3.3.3. Example data
  • 3.3.4. Input/output, data types and colorspaces
    • 3.3.4.1. Data types
    • 3.3.4.2. Colorspaces
  • 3.3.5. Image preprocessing / enhancement
    • 3.3.5.1. Local filters
    • 3.3.5.2. Non-local filters
    • 3.3.5.3. Mathematical morphology
  • 3.3.6. Image segmentation
    • 3.3.6.1. Binary segmentation: foreground + background
    • 3.3.6.2. Marker based methods
  • 3.3.7. Measuring regions’ properties
  • 3.3.8. Data visualization and interaction
  • 3.3.9. Feature extraction for computer vision
  • 3.3.10. Full code examples
  • 3.3.11. Examples for the scikit-image chapter
• 3.4. scikit-learn: machine learning in Python
  • 3.4.1. Introduction: problem settings
    • 3.4.1.1. What is machine learning?
    • 3.4.1.2. Data in scikit-learn
  • 3.4.2. Basic principles of machine learning with scikit-learn
    • 3.4.2.1. Introducing the scikit-learn estimator object
    • 3.4.2.2. Supervised Learning: Classification and regression
    • 3.4.2.3. A recap on Scikit-learn’s estimator interface
    • 3.4.2.4. Regularization: what it is and why it is necessary
  • 3.4.3. Supervised Learning: Classification of Handwritten Digits
    • 3.4.3.1. The nature of the data
    • 3.4.3.2. Visualizing the Data on its principal components
    • 3.4.3.3. Gaussian Naive Bayes Classification
    • 3.4.3.4. Quantitative Measurement of Performance
  • 3.4.4. Supervised Learning: Regression of Housing Data
    • 3.4.4.1. A quick look at the data
    • 3.4.4.2. Predicting Home Prices: a Simple Linear Regression
  • 3.4.5. Measuring prediction performance
    • 3.4.5.1. A quick test on the K-neighbors classifier
    • 3.4.5.2. A correct approach: Using a validation set
    • 3.4.5.3. Model Selection via Validation
    • 3.4.5.4. Cross-validation
    • 3.4.5.5. Hyperparameter optimization with cross-validation
  • 3.4.6. Unsupervised Learning: Dimensionality Reduction and Visualization
    • 3.4.6.1. Dimensionality Reduction: PCA
    • 3.4.6.2. Visualization with a non-linear embedding: tSNE
  • 3.4.7. Parameter selection, Validation, and Testing
    • 3.4.7.1. Hyperparameters, Over-fitting, and Under-fitting
    • 3.4.7.2. Visualizing the Bias/Variance Tradeoff
    • 3.4.7.3. Summary on model selection
    • 3.4.7.4. A last word of caution: separate validation and test set
  • 3.4.8. Examples for the scikit-learn chapter