Key points to understand about computer vision include:
Computer vision is accomplished by using large numbers of images to train a model.
Image classification is a form of computer vision in which a model is trained with images that are labeled with the main subject of the image (in other words, what it’s an image of) so that it can analyze unlabeled images and predict the most appropriate label – identifying the subject of the image.
Object detection is a form of computer vision in which the model is trained to identify the location of specific objects in an image.
There are more advanced forms of computer vision – for example, semantic segmentation is an advanced form of object detection where, rather than indicate an object’s location by drawing a box around it, the model can identify the individual pixels in the image that belong to a particular object.
You can combine computer vision and language models to create a multi-modal model that combines computer vision and generative AI capabilities.
Computer vision scenarios
Common uses of computer vision include:
Auto-captioning or tag-generation for photographs.
Visual search.
Monitoring stock levels or identifying items for checkout in retail scenarios.
Key points to understand about generative AI include:
Generative AI is a branch of AI that enables software applications to generate new content; often natural language dialogs, but also images, video, code, and other formats.
The ability to generate content is based on a language model, which has been trained with huge volumes of data – often documents from the Internet or other public sources of information.
Generative AI models encapsulate semantic relationships between language elements (that’s a fancy way of saying that the models “know” how words relate to one another), and that’s what enables them to generate a meaningful sequence of text.
There are large language models (LLMs) and small language models (SLMs) – the difference is based on the volume of data and the number of variables in the model. LLMs are very powerful and generalize well, but can be more costly to train and use. SLMs tend to work well in scenarios that are more focused on specific topic areas, and usually cost less.
Generative AI scenarios
Common uses of generative AI include:
Implementing chatbots and AI agents that assist human users.
Creating new documents or other content (often as a starting point for further iterative development)
Now that you understand basic KQL querying and optimization techniques, let’s explore materialized views and stored functions in eventhouses.
Understand materialized views
Materialized views are precomputed aggregations that solve a common performance challenge in KQL databases. KQL databases in eventhouses often contain millions or billions of rows from streaming data sources like IoT sensors, application logs, and other events. Running aggregation queries across these large datasets can take significant time and computing resources.
Materialized views store precomputed aggregation results and automatically update them as new data arrives. Instead of recalculating metrics from all historical data every time you query, the materialized view maintains the results and only processes the new data to update the aggregations. This provides instant results for dashboards and reports, even when working with massive datasets.
How automatic updates work
A materialized view consists of two parts that work together to provide always-current results:
A materialized part: Precomputed aggregation results from data that has already been processed
A delta: New data that has arrived since the last background update
When you query a materialized view, the system automatically combines both parts at query time to give you fresh, up-to-date results. This means materialized views always return current data, regardless of when the background materialization process last ran. Meanwhile, a background process periodically moves data from the delta part into the materialized part, keeping the precomputed results current. This approach provides the speed of precomputed results with the freshness of real-time data.
Create materialized views
A materialized view encapsulates a KQL summarize statement that automatically updates as new data arrives. Here’s an example that tracks trip metrics by vendor and day:
Once created, materialized views can be queried like regular tables:
kqlCopy
TripsByVendor
| where pickup_date >= ago(7d)
| project pickup_date, vendor_id, trips, avg_fare, total_revenue
| sort by pickup_date desc, total_revenue desc
Understand stored functions
KQL includes the ability to encapsulate a query as a function, making it easier to repeat common queries. You can also specify parameters for a function, so you can repeat the same query with variable values.
Stored functions are useful in eventhouses where you have streaming data and multiple people writing queries. Instead of writing the same filtering or transformation logic repeatedly, you can define it once as a function and reuse it across different queries. Functions also help ensure that calculations are performed consistently when different team members need to apply the same logic to the data.
Create a function
kqlCopy
.create-or-alter function trips_by_min_passenger_count(num_passengers:long)
{
TaxiTrips
| where passenger_count >= num_passengers
| project trip_id, pickup_datetime
}
To call the function, use it like a table. In this example, the trips_by_min_passenger_count function is used to find 10 trips with at least three passengers:
Understanding how to write efficient KQL queries is essential for getting good performance when working with Eventhouses. This unit covers key optimization techniques and explains why they matter for your queries.
Why query optimization matters
Query performance in KQL databases depends on the amount of data processed. When you understand how KQL processes data, you can write queries that:
Run faster by reducing the data scanned – For example, instead of scanning millions of rows, filter early to process only thousands
Use fewer resources – For example, selecting only 3 columns instead of all 50 columns reduces processing overhead
Work reliably with growing data – For example, a query that works on 1 million rows today will still perform well when your data grows to 10 million rows
The key principle is: the less data your query needs to process, the faster it runs.
Understand key optimization techniques
Filter data early and effectively
Filtering reduces the amount of data that subsequent operations need to process, and KQL databases use indexes and data organization techniques that make early filtering especially efficient.
Time-based filtering is effective because Eventhouses typically contain time-series data:
kqlCopy
TaxiTrips
| where pickup_datetime > ago(30min) // Filter first - uses time index
| project trip_id, vendor_id, pickup_datetime, fare_amount
| summarize avg_fare = avg(fare_amount) by vendor_id
Order your filters by how much data they eliminate – put filters that eliminate the most data first. Think of it like a funnel: start with the filter that removes the most rows, then apply more specific filters to the remaining data:
kqlCopy
TaxiTrips
| where pickup_datetime > ago(1d) // Time filter first - eliminates most data
| where vendor_id == "VTS" // Specific vendor - eliminates some data
| where fare_amount > 0 // Value filter - eliminates least data
| summarize trip_count = count()
Reduce columns early
Projecting or selecting only the columns you need reduces resource usage. This is especially important when working with wide tables that have many columns.
kqlCopy
TaxiTrips
| project trip_id, pickup_datetime, fare_amount // Select columns early
| where pickup_datetime > ago(1d) // Then filter
| summarize avg_fare = avg(fare_amount)
Optimize aggregations and joins
Aggregations and joins are resource-intensive operations because they need to process and combine large amounts of data. How you structure them can significantly affect query performance.
For aggregations, limit results when exploring data:
kqlCopy
TaxiTrips
| where pickup_datetime > ago(1d)
| summarize trip_count = count() by trip_id, vendor_id
| limit 1000 // Limit results for exploration
For joins, put the smaller table first. When joining tables, KQL processes the first table to match with the second table. Starting with a smaller table means fewer rows to process, making the join more efficient.
kqlCopy
// Good: Small vendor table first
VendorInfo
| join kind=inner TaxiTrips on vendor_id
// Avoid: Large taxi table first
TaxiTrips
| join kind=inner VendorInfo on vendor_id
When you create an Eventhouse, a default KQL database is automatically created with the same name. An Eventhouse contains one or more KQL databases, where you can create tables, stored procedures, materialized views, functions, data streams, and shortcuts to manage your data. You can use the default KQL database or create other KQL databases as needed.
Work with data in your Eventhouse
There are several ways to access and work with data in a KQL database within an Eventhouse:
Data ingestion
You can ingest data directly into your KQL database from various sources:
Connectors to sources such as Apache Kafka, Confluent Cloud Kafka, Apache Flink, MQTT (Message Queuing Telemetry Transport), Amazon Kinesis, Google Cloud Pub/Sub
Database shortcuts
You can create database shortcuts to existing KQL databases in other eventhouses or Azure Data Explorer databases. These shortcuts let you query data from external KQL databases as if the data were stored locally in your eventhouse, without actually copying the data.
OneLake availability
You can enable OneLake availability for individual KQL databases or tables, making your data accessible throughout the Fabric ecosystem for cross-workload integration with Power BI, Warehouse, Lakehouse, and other Fabric services.
Query data in a KQL database
To query data in a KQL database, you can use KQL or T-SQL in KQL querysets. When you create a KQL database, an attached KQL queryset is automatically created for running and saving queries.
Basic KQL syntax
KQL uses a pipeline approach where data flows from one operation to the next using the pipe (|) character. Think of it like a funnel – you start with an entire data table, and each operator filters, rearranges, or summarizes the data before passing it to the next step. The order of operators matters because each step works on the results from the previous step.
Important
KQL is case-sensitive for everything including table names, column names, function names, operators, keywords, and string values. All identifiers must match exactly. For example, TaxiTrips is different from taxitrips or TAXITRIPS.
Here’s an example that shows the funnel concept:
kqlCopy
TaxiTrips
| where fare_amount > 20
| project trip_id, pickup_datetime, fare_amount
| take 10
This query starts with all data in the TaxiTrips table, filters it to show only trips with fares over $20, selects specific columns using the project operator, and uses the take operator to return the first 10 rows that match the criteria in the where clause.
The simplest KQL query consists of a table name:
kqlCopy
TaxiTrips
This returns all columns from the TaxiTrips table, but the number of rows displayed is limited by your query tool’s default settings.
To retrieve a sample of data from potentially large tables, use the take operator:
kqlCopy
TaxiTrips
| take 100
This returns the first 100 rows from the TaxiTrips table, which is useful for exploring data structure without processing the entire table.
You can also aggregate data:
kqlCopy
TaxiTrips
| summarize trip_count = count() by taxi_id
This returns a summary table showing the total number of trips (trip_count) for each unique taxi_id, effectively counting how many trips each taxi has made.
Analyze data with KQL queryset
KQL queryset provides a workspace for running and managing queries against KQL databases. The KQL queryset allows you to save queries for future use, organize multiple query tabs, and share queries with others for collaboration. The KQL queryset also supports T-SQL queries, allowing you to use T-SQL syntax alongside KQL for data analysis.
You can also create data visualizations while exploring your data, rendering query results as charts, tables, and other visual formats.
Use Copilot to assist with queries
For AI-based assistance with KQL querying, you can use Copilot for Real-Time Intelligence
When your administrator enables Copilot, you see the option in the queryset menu bar. Copilot opens as a pane to the side of the main query interface. When you ask a question about your data, Copilot generates the KQL code to answer your question.
Throughout history, technological advancements influenced the way we work. Today, AI is leading a new wave of change by introducing innovative ways to manage workplace tasks and redefining economic opportunities.
The video explores how AI creates these new opportunities, highlighting its potential to enhance your professional goals and everyday tasks. From automating routine tasks to giving personalized career advice, AI is changing the landscape. Whether you’re aiming to improve your current role or explore new career directions, AI offers tools and insights to help you navigate these opportunities.
https://go.microsoft.com/fwlink/?linkid=2287800
Generative AI isn’t just a tool for automation, but a powerful ally in enhancing creativity, problem-solving, and career development.
Generative AI offers various technologies to assist in idea creation. This video will cover a few examples, such as text-based generation, text-to-image generation, audio generation, and video generation. Each technology offers unique ways to bring your ideas to life.
For example, text-based generation can help draft content or brainstorm new concepts. Text-to-image generation can transform your textual descriptions into vivid images. Audio generation can create music or sound effects based on your prompts. Video generation can produce visual content from text, simplifying the creation of engaging videos without requiring extensive editing skills.
The video will show how you can use these technologies today, highlighting their potential to enhance creativity and streamline the idea creation process. Whether you’re exploring new ideas or bringing your visions to life, generative AI technology can significantly expand your creative toolkit.
https://go.microsoft.com/fwlink/?linkid=2287701
Generative AI technology offers a fresh set of tools for creativity, enabling you to push the limits of your aspirations and goals.
With generative AI, visualizing ideas is more accessible than ever. One of the most exciting advancements is its ability to transform text descriptions into images. This technology, known as text-to-image generation, uses AI models to interpret text into visual representations. Imagine you’ve written a short story and want illustrations to bring it to life. By describing a scene through text, AI can create illustrations that match your description in no time, helping you find inspiration quickly or even critically analyze your story visually to see if you want to make any changes to enhance it further.
This technology isn’t limited to authors. Designers, marketers, educators, and anyone with a creative vision can benefit from it. For instance, a teacher could describe a historical event and have AI generate an image to make the lesson more engaging for students. A marketer could visualize a campaign concept before it’s executed. The possibilities are endless.
This video will introduce you to text-to-image technology and how you can access it today in Microsoft Copilot. By leveraging tools like DALL·E, integrated into Microsoft Designer, you can turn your textual descriptions into vivid images, bridging the gap between imagination and reality. Whether you’re looking to enhance a story, create visuals for a project, or simply explore your creativity, text-to-image AI is a powerful tool at your disposal.
https://go.microsoft.com/fwlink/?linkid=2287802
Generative AI is unlocking new possibilities for the creative process. With text-to-image generation, you can bring your ideas to life visually. In the next lesson, you’ll learn how AI can further assist you in content creation, making it an invaluable companion in your creative journey.
You might come across some interesting AI acronyms such as Large Language Models (LLMs) or Natural Language Generation (NLG). These acronyms are a part of a branch of AI called Natural Language Processing (NLP).
These technologies enable computers to understand, generate, and respond to human language in new ways. From suggesting words while typing a message, to giving ideas for a creative project, AI provides further support for our creative endeavors.
The video explains the meaning behind the acronyms and how this branch of AI is changing how humans interact with AI.
https://go.microsoft.com/fwlink/?linkid=2287801
Natural Language Processing (NLP) is a branch of AI that helps computers understand and respond to human language. Recently, Large Language Models (LLMs) improved NLP applications.
LLMs are advanced AI models trained in large amounts of text data. They predict words in sequences, enabling them to perform tasks like text generation, summarization, translation, and classification.
LLMs are a key part of generative AI, which focuses on creating new content. By using LLMs, generative AI can produce human-like text, making it a useful tool in content creation.
Generative AI is transforming the approach to productivity. Recent advancements in AI greatly improved natural language processing and generation. These technologies now enable the creation of images, videos, texts, and audio from simple descriptions, transforming how you interact with technology.
This video reveals how generative AI can provide support in the creative process and enhance productivity, highlighting its potential to reshape various industries by empowering people to achieve more with less. By automating repetitive tasks and providing creative suggestions, generative AI allows you to focus on what truly matters: envisioning innovative ideas, setting ambitious goals, and pursuing your dreams.
https://go.microsoft.com/fwlink/?linkid=2287901
Now that you have the foundations down, you might be curious about how it works. Generative AI functions through AI models, which are mathematical structures that learn from patterns in data using algorithms. There are several types of AI models with varying capabilities. Some AI models are designed to identify and classify information, while others, like generative AI models, excel in creating content.
This video shows how the use of generative AI models varies based on one’s technical ability. For example, experts can customize these models for complex tasks, while beginners can use preexisting models or tools with minimal technical knowledge.
https://go.microsoft.com/fwlink/?linkid=2287900
Generative AI is changing the way you interact with technology, from creating content to helping with complex tasks. These advancements enable AI to understand and process human language, fostering a more natural interaction.
