Książki

This unique open access book applies the functional OCaml programming language to numerical or computational weighted data science, engineering, and scientific applications. This book is based on the authors' first-hand experience building and maintaining Owl, an OCaml-based numerical computing library. 

You'll first learn the various components in a modern numerical computation library. Then, you will learn how these components are designed and built up and how to optimize their performance. After reading and using this book, you'll have the knowledge required to design and build real-world complex systems that effectively leverage the advantages of the OCaml functional programming language.

What You Will Learn

• Optimize core operations based on N-dimensional arrays
• Design and implement an industry-level algorithmic differentiation module
• Implement mathematical optimization, regression, and deep neural network functionalities based on algorithmic differentiation
• Design and optimize a computation graph module, and understand the benefits it brings to the numerical computing library
• Accommodate the growing number of hardware accelerators (e.g. GPU, TPU) and execution backends (e.g. web browser, unikernel) of numerical computation
• Use the Zoo system for efficient scripting, code sharing, service deployment, and composition
• Design and implement a distributed computing engine to work with a numerical computing library, providing convenient APIs and high performance

Who This Book Is For
Those with prior programming experience, especially with the OCaml programming language, or with scientific computing experience who may be new to OCaml. Most importantly, it is for those who are eager to understand not only how to use something, but also how it is built up.

Architecture of Advanced Numerical Analysis Systems: Designing a Scientific Computing System using OCaml

PrologueA Brief HistoryReductionism vs. HolismKey FeaturesContact MePART 1: NUMERICAL TECHNIQUES1. IntroductionWhat Is Scientific ComputingWhat is Functional ProgrammingWho Is This Book ForStructure of the BookInstallationOption 1: Install from OPAMOption 2: Pull from Docker HubOption 3: Pin the Dev-RepoOption 4: Compile from SourceCBLAS/LAPACKE DependencyInteracting with OwlUsing ToplevelUsing NotebookUsing Owl-JupyterSummary2. ConventionsPure vs. ImpureNdarray vs. ScalarInfix OperatorsOperator ExtensionModule StructuresNumber and PrecisionPolymorphic FunctionsModule ShortcutsType Casting3. VisualisationCreate PlotsSpecificationSubplotsMultiple LinesLegendDrawing PatternsLine PlotScatter PlotStairs PlotBox PlotStem PlotArea PlotHistogram & CDF PlotLog Plot3D PlotAdvanced Statistical PlotSummaryReferences4. Mathematical FunctionsBasic FunctionsBasic Unary Math FunctionsBasic Binary FunctionsExponential and Logarithmic FunctionsTrigonometric FunctionsOther Math FunctionsSpecial FunctionsAiry FunctionsBessel FunctionsElliptic FunctionsGamma FunctionsBeta FunctionsStruve FunctionsZeta FunctionsError FunctionsIntegral FunctionsFactorialsInterpolation and ExtrapolationIntegrationUtility FunctionsSummary5. Statistical FunctionsRandom VariablesDiscrete Random VariablesContinuous Random VariablesDescriptive StatisticsOrder StatisticsSpecial DistributionGamma DistributionBeta DistributionChi-Square DistributionStudent-t DistributionCauchy DistributionMultiple VariablesSamplingHypothesis TestsTheoryGaussian Distribution in Hypothesis TestingTwo-Sample InferencesGoodness-of-fit TestsNon-parametric StatisticsCovariance and CorrelationsAnalysis of VarianceSummary6. N-Dimensional ArraysNdarray TypesCreation FunctionsProperties FunctionsMap FunctionsFold FunctionsScan FunctionsComparison FunctionsVectorised FunctionsIteration FunctionsManipulation FunctionsSerialisationTensorsSummaryReferences7. Slicing and BroadcastingSlicingBasic SlicingFancy SlicingConventions in DefinitionExtended OperatorsAdvanced UsageBroadcastingWhat Is Broadcasting?Shape ConstraintsSupported OperationsSlicing in NumPy and JuliaInternal MechanismSummary8. Linear AlgebraVectors and MatricesCreating MatricesAccessing ElementsIterate, Map, Fold, and FilterMath OperationsGaussian EliminationLU FactorisationInverse and TransposeVector SpacesRank and BasisOrthogonalitySolving Ax = bMatrix SensitivityDeterminantsEigenvalues and EigenvectorsSolving Ax=  xComplex MatricesSimilarity Transformation and DiagonalisationPositive Definite MatricesPositive DefinitenessSingular Value DecompositionInternal: CBLAS and LAPACKELow-level Interface to CBLAS & LAPACKESparse MatricesSummaryReferences9. Ordinary Differential EquationsWhat Is An ODEExact SolutionsLinear SystemsSolving An ODE NumericallyOwl-ODEExample: Linear Oscillator SystemSolver StructureSymplectic SolversFeatures and LimitsExamples of using Owl-ODEExplicit ODETwo Body ProblemLorenz AttractorDamped OscillationStiffnessSolve Non-Stiff ODEsSolve Stiff ODEsSummaryReferences10. Signal ProcessingDiscrete Fourier TransformFast Fourier TransformExamplesApplications of FFTFind period of sunspotsDecipher the ToneImage ProcessingFilteringExample: SmoothingGaussian FilterSignal ConvolutionFFT and Image ConvolutionSummaryReferences11. Algorithmic DifferentiationChain RuleDifferentiation MethodsHow Algorithmic Differentiation WorksForward ModeReverse ModeForward or Reverse?A Strawman AD EngineSimple Forward ImplementationSimple Reverse ImplementationUnified ImplementationsForward and Reverse Propagation APIExpressing ComputationExample: Forward ModeExample: Reverse ModeHigh-Level APIsDerivative and GradientJacobianHessian and LaplacianOther APIsInternal of Algorithmic DifferentiationGo Beyond Simple ImplementationExtend AD moduleLazy EvaluationSummaryReferences12. OptimisationIntroductionRoot FindingUnivariate Function OptimisationUse DerivativesGolden Section SearchMultivariate Function OptimisationNelder-Mead Simplex MethodGradient Descent MethodsConjugate Gradient MethodNewton and Quasi-Newton MethodsGlobal Optimisation and Constrained OptimisationSummaryReferences13. RegressionLinear RegressionProblem: Where to locate a new McDonald's restaurant?Cost FunctionSolving Problem with Gradient DescentMultiple RegressionFeature NormalisationAnalytical SolutionNon-linear regressionsRegularisationOls, Ridge, Lasso, and Elastic_netLogistic RegressionSigmoid FunctionCost FunctionExampleMulti-class classificationSupport Vector MachineKernel and Non-linear BoundaryExampleModel error and selectionError MetricsModel SelectionSummaryReferences14. Deep Neural NetworksPerceptronYet Another RegressionModel RepresentationForward PropagationBack propagationFeed Forward NetworkLayersActivation FunctionsInitialisationTrainingTestNeural Network ModuleModule StructureNeuronsNeural GraphTraining ParametersConvolutional Neural NetworkRecurrent Neural NetworkLong Short Term Memory (LSTM)Generative Adversarial NetworkSummaryReferences15. Natural Language ProcessingIntroductionText CorpusStep-by-step OperationUse the Corpus ModuleVector Space ModelsBag of Words (BOW)Term Frequency-Inverse Document Frequency (TF-IDF)Latent Dirichlet Allocation (LDA)ModelsDirichlet DistributionGibbs SamplingTopic Modelling ExampleLatent Semantic Analysis (LSA)Search Relevant DocumentsEuclidean and Cosine SimilarityLinear SearchingSummaryReferences16. Dataframe for Tabular DataBasic ConceptsCreate FramesManipulate FramesQuery FramesIterate, Map, and FilterRead/Write CSV FilesInfer Type and SeparatorSummary17. Symbolic RepresentationIntroductionDesignCore abstractionEnginesONNX EngineExample 1: Basic operationsExample 2: Variable InitialisationExample 3: Neural networkLaTeX EngineOwl EngineSummary18. Probabilistic ProgrammingGenerative Model vs Discriminative ModelBayesian NetworksSampling TechniquesInferencePART 2: SYSTEM ARCHITECTURE19. Architecture OverviewIntroductionArchitecture OverviewCore ImplementationN-dimensional ArrayInterfaced LibrariesAdvanced FunctionalityComputation GraphAlgorithmic DifferentiationRegressionNeural NetworkParallel ComputingActor EngineGPU ComputingOpenMPCommunity-Driven R&DSummary20. Core OptimisationBackgroundNumerical LibrariesOptimisation of Numerical ComputationInterfacing to C CodeNdarray OperationsFrom OCaml to COptimisation TechniquesMap OperationsConvolution OperationsReduction OperationsRepeat OperationsSummaryReferences21. Automatic Empirical TuningWhat is Parameter TuningWhy Parameter Tuning in OwlHow to Tune OpenMP ParametersMake a DifferenceSummary22. Computation GraphIntroductionWhat is a Computation Graph?From Dynamic to StaticSignificance in ComputingExamplesExample 01: Basic CGraphExample 02: CGraph with ADExample 03: CGraph with DNNDesign RationaleOptimisation of CGraphOptimising memory with pebblesAllocation AlgorithmAs Intermediate RepresentationsSummary23. Scripting and Zoo SystemIntroductionShare Script with ZooTypical ScenarioCreate a ScriptShare via GistImport in Another ScriptSelect a Specific VersionCommand Line ToolMore ExamplesSystem DesignServicesType CheckingBackendDomain Specific LanguageService DiscoveryUse CaseSummaryReferences24. Compiler BackendsBase LibraryBackend: JavaScriptUse Native OCamlUse Facebook ReasonBackend: MirageOSMirageOS and UnikernelExample: Gradient DescentExample: Neural NetworkEvaluationSummary25. Distributed ComputingActor SystemDesignActor EnginesMap-Reduce EngineParameter Server EnginePeer-to-Peer EngineClassic Synchronise ParallelBulk Synchronous ParallelAsynchronous ParallelStale Synchronous ParallelProbabilistic Synchronise ParallelBasic idea: samplingCompatibilityBarrier Trade-off DimensionsConvergenceA Distributed Training ExampleStep ProgressAccuracySummaryReferences26. Testing FrameworkUnit TestExampleWhat Could Go WrongCorner CasesTest CoverageUse FunctorSummary27. Constants and Metric SystemWhat Is a Metric SystemFour Metric SystemsSI PrefixExample: Physics and Math constantsInternational System of UnitsTimeLengthAreaVolumeSpeedMassForceEnergyPowerPressureViscosityLuminanceRadioactivity28. Internal Utility ModulesDataset ModuleMNISTCIFAR-10Graph ModuleStack and Heap ModulesCount-Min SketchSummaryPART 3: CASE STUDIES29. Case - Image RecognitionBackgroundLeNetAlexNetVGGResNetSqueezeNetCapsule NetworkBuilding InceptionV3 NetworkInceptionV1 and InceptionV2FactorisationGrid Size ReductionInceptionV3 ArchitecturePreparing WeightsProcessing ImageRunning InferenceApplicationsSummaryReferences30. Case - Instance SegmentationIntroductionMask R-CNN NetworkBuilding Mask R-CNNFeature ExtractorProposal GenerationClassificationRun the CodeSummaryReferences31. Case - Neural Style TransferContent and StyleContent ReconstructionStyle RecreationCombining Content and StyleRunning NSTExtending NSTFast Style TransferBuilding FST NetworkRunning FSTSummaryReferences32. Case - Recommender SystemIntroductionArchitectureBuild Topic ModelsIndex Text CorpusRandom ProjectionOptimising Vector StorageOptimise Data StructureOptimise Index AlgorithmSearch ArticlesCode ImplementationMake It LiveSummaryReferences33. Case - Applications in FinanceIntroductionBond PricingBlack-Scholes ModelMathematical ModelOption PricingPortfolio OptimisationMathematical ModelEfficient FrontierMaximise Sharpe Ratio