Data Engineering and Data Warehousing Data Science

Machine learning

Machine learning is a category of algorithm that make the software application more accurate in predicting outcomes without being explicitly programmed .
The basic premise of machine learning is to build algorithms that can receive input data and use statical analysis to predict an output while updating outputs as new data becomes available.

Machine Learning was born from pattern recognition and the theory that computers can learn without being programmed to perform specific tasks . the good thing about Machine learning is the ability to independently adapt because it learns from the past computations to produce reliable , repeatable decisions and results .

What’s required to create good machine learning systems?

  • Data preparation capabilities.
  • Algorithms – basic and advanced.
  • Automation and iterative processes.
  • Scalability.
  • Ensemble modeling.

Data Engineering and Data Warehousing Data Science

What is Data Warehouse ?

In many organizations, we want a central “store” of all of our entities, concepts, metadata, and historical information . For doing data validation, complex mining, analysis, prediction, … and this is the data warehouse.

To be more precise , data warehouse is simply a single, complete, and consistent store of data obtained from a variety of sources and made available to end users in a way they can understand and use it in a business context.”

One of the “modern” uses of the data warehouse is not only to support analytics but to serve as a reference to all of the entities in the organization.

In breif , Data warehouse is a collection of data that is used primarily in organizational decision making .And In order for data to be effective, Data Warehouse must be: Consistent ,Well integrated ,Well defined and Time stamped.