探索 Metavision® 智能套件

荣获五十余项大奖的事件视觉软件工具包。

 

免费下载,免费制作,免费销售

 

 

荣获五十余项大奖的事件视觉软件套件

事件视觉软件套件提供了超过95种算法、67个代码示例和11个即用型的应用,可供您构建自己的基于事件的视觉产品。该套件屡获殊荣,是迄今为止业界广泛的选择
开源架构

元视觉智能套件基于开源架构 ,使得软件和硬件设备之间完全互通,并打造了一个快速增长的基于事件的社区。

了解更多

领先的基于事件的机器学习工具包

利用NeurIPS上备受关注,迄今为止高效的物体检测器庞大的高清数据集、事件到视频和视频到事件管道、训练、推理、评价功能等,构建先进的基于事件的机器学习网络。

了解更多

 

6大系列模块

广泛覆盖的计算机视觉的各个领域:机器学习,计算机视觉,相机标定,高性能应用等等,您正在寻找的工具就在这里。

大量的文档资料和支持服务

docs.prophesee.ai 上有300多页定期更新的内容,20多个Jupyter notebook大量的参考数据和各类指南,让您的产品开发有一个良好的开端。

快速获得结果

Prophesee花了6年多的时间来完善这一庞大的预构建管道、广泛的数据集、代码示例和GUI工具等,确保您可以在几分钟内得到结果。

免费下载、免费搭建、免费商业化授权

 

Prophesee的软件工具包可以在PC端Windows和Ubuntu系统中完全免费使用,没有任何时间限制。它包括一个商业级的许可证,使您能够利用Prophesee 基于事件的元视觉传感技术,可用于设计和销售自己基于元视觉智能的产品

软件

工具

硬件

软件

检测推理

一套专门的工具为您提供了使用事件执行深度神经网络(DNN)所需的一切,以便释放基于事件的机器学习的潜力。您可以充分利用我们用pytorch编写的预训练的汽车模型,使用我们的c++管道进行实时检测和跟踪实验。使用我们功能全面的python库,来设计您自己的网络。

在15小时长度,拥有2300万标签的汽车数据集上训练的预训练网络

实时检测和跟踪 @100Hz

检测训练

您可以随时使用我们的训练框架训练您自己的物体检测应用。您可以实验多个预先构建的基于事件的张量表示,训练适合事件的数据的网络。

4 个预建的张量表示

自动生成的 HDF5 数据集 

功能全面的训练工具箱,包括量身定做的预处理、DataLoader、NN架构、可视化工具等。

视频到事件

视频到事件管道连接基于帧和基于事件的世界。生成合成数据以扩充数据集,并部分重复使用现有的资料。

现成的、随时可用的事件模拟器

无缝集成与GPU兼容的事件模拟器 

事件到视频

使用该神经网络根据事件构建灰度图像。对于基于帧方案已解决的任务,可以借此来做事件数据的自动注释,以节省资源。

现成的 随时可用的事件到视频转换器

充分利用现有的基于帧的神经网络 来获得标签

 

 

经检测KPI

用我们符合最新COCO API标准的物体检测KPI工具包来评估您的检测性能。

包括 mAP, mAR 和它们的变体

光流推断

利用我们预先训练的流模型,从基于事件的数据中预测光流;利用我们预先训练的光流模型、定制的数据加载器以及损失函数和可视化工具集合来建立你的光流预测管道。

自监督的 Flownet架构 

轻量级模型

光流训练

没有真实值?充分利用我们的自监督架构。用我们定制的FlowNet训练框架来训练您的光流应用,为您基于事件的数据量身定制4个预先构建的流网络

功能全面的光流训练工具箱括不同的网络拓扑结构、各种损失函数和可视化模式

手势分类推理

用我们预先训练好的模型在事件相机或事件数据中运行一个实时的剪刀石头布游戏

现成的可随时使用的分类器

手势分类训练

用这个量身定做的基于事件的训练管道来训练您的物体分类器

全面的训练工具箱,包括量身定做的预处理、DataLoader、NN架构、可视化工具等。

振动监测

通过跟踪场景中每个像素的时序信号,以像素级的精度连续、远程地监测振动频率。每个事件,像素坐标、变化的极性和准确的时间戳都被记录下来,从而提供对振动模式的全面、连续的理解

1Hz 到 kHz 范围

1 个像素 的精度

飞溅物监测

追踪具有溅射状运动的小颗粒。事件的视觉传感器由于具有高时间分辨率和动态范围,可以在最苛刻的环境中追踪小颗粒。

高达 200kHz 的跟踪频率 (5µs 的时间分辨率)

>同时对所有颗粒进行XYT追踪  

高速计数

以高速对物体进行计数,精确度高,产生的数据少没有任何运动模糊。物体在经过视野时被计数,当有物体运动时,相应像素才会被触发

吞吐量:>1,000 个物体/秒。

精度:>99.5% @1,000 个物体/秒。

粒径监测

控制、计数和测量在通道或传送带上以极高速度移动的物体的尺寸,以便在您的生产线上获得即时的质量统计数据,以控制生产过程。

高达 500,000 像素/秒 的速度

99.9% 的计数精度

物体追踪

追踪视野中的移动物体,利用基于事件的传感器具备的低数据率和稀疏化特性,在低算力条件下跟踪物体。

时间上的连续跟踪:在先后拍摄到的两帧之间不再有 “盲点”。

原生分割 只分析运动,忽略静态背景

标定

在现实环境中部署应用程序,并控制基于事件的系统的所有光学参数。我们提供一整套预置的工具来标定您的相机和调整焦点。根据您的具体需要进行扩展,并使用我们的算法模块连接到标准标定程序

镜头对焦评估

自动相机内参标定

 

边缘跟踪

为您的AR/VR应用追踪3D边缘和/或基准点。受益于事件的高时间分辨率,可以提高您的边缘跟踪应用的准确性和鲁棒性。

使用几何先验条件的自动3D物体检测

三维物体实时跟踪  

光流

通过事件,重现审视这一计算机视觉基础构建模块。颠覆传统的逐帧分析方法,利用连续的逐个像素的跟踪,更有效地理解运动。

与传统的基于图像的方法相比,功耗降低17倍

只在移动物体上采集特征点。

XYT 视觉化

通过我们的XYT查看器将您的数据可视化,探索时间-空间连续性对您应用的效用。

发现帧之间的变化

放大时间,了解场景中的运动

 

数据率可视化

了解事件随时间产生的过程,利用python的功能实现数据的可视化并生成图表。

随时可用的 Python环境

与数据文件和事件相机的接口

超慢动作

放慢时间,慢到时间分辨率相当于每秒超过20万帧,而生成的数据比传统方法少了几个数量级。帮助了解隐藏在超高速的和极短暂的事件中的精细的运动动态

高达 200,000 fps(等效时间分辨率)

工具

无论你是否拥有EVK,Metavision Studio都可以作为您完美的启动工具

其具有图形用户界面,任何人都可通过与PROPHESEE兼容的基于事件的视觉系统进行可视化和记录数据流。

它还能够使您浏览其所提供的事件数据集,以加深对基于事件的视觉的理解

 

可视化、信息流、调整事件

全面的图形用户界面

主要特点
  • 来自与Prophesee兼容的基于事件的视觉系统的数据可视化
  • 数据可视化控制(累积时间,fps)
  • 数据记录
  • 回放记录数据
  • 导出为AVI视频
  • 控制,保存和加载传感器设置
无论你是否拥有EVK,Metavision Studio都可以作为您完美的启动工具

其具有图形用户界面,任何人都可通过与PROPHESEE兼容的基于事件的视觉系统进行可视化和记录数据流。

它还能够使您浏览其所提供的事件数据集,以加深对基于事件的视觉的理解

 

可视化、信息流、调整事件

全面的图形用户界面

主要特点
  • 来自与Prophesee兼容的基于事件的视觉系统的数据可视化
  • 数据可视化控制(累积时间,fps)
  • 数据记录
  • 回放记录数据
  • 导出为AVI视频
  • 控制,保存和加载传感器设置

Metavision SDK is the largest set of Event-Based Vision algorithms accessible to date. High-performance algorithms are available via APIs, ready to go to production with Event-Based Vision applications.

The provided algorithms are available as C++ APIs for highly efficient applications and runtime execution or Python APIs giving access to the C++ algorithms, Machine Learning pipelines and more.

Develop high-performance Event-Based Vision solutions

25 c++ code samples, 44 python classes 51 algorithms, 26 python samples, 16 tutorials and 11 ready-to-use applications

Main features
  • Extensive documentation / code examples /  training & learning material available
  • Runs natively on Linux and Windows
  • Compatible with Prophesee vision systems and « Powered by Prophesee » partners products
  • A complete API that allows you to explore the full potential of Event-Based Vision in just a few lines of code
/**********************************************************************************************************************
 * Copyright (c) Prophesee S.A.                                                                                       *
 *                                                                                                                    *
 * Licensed under the Apache License, Version 2.0 (the "License");                                                    *
 * you may not use this file except in compliance with the License.                                                   *
 * You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0                                 *
 * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed   *
 * on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.                      *
 * See the License for the specific language governing permissions and limitations under the License.                 *
 **********************************************************************************************************************/
​
// This code sample demonstrate how to use the Metavision C++ SDK. The goal of this sample is to create a simple event
// counter and displayer by introducing some basic concepts of the Metavision SDK.
​
#include <metavision/sdk/driver/camera.h>
#include <metavision/sdk/base/events/event_cd.h>
#include <metavision/sdk/core/algorithms/periodic_frame_generation_algorithm.h>
#include <metavision/sdk/ui/utils/window.h>
#include <metavision/sdk/ui/utils/event_loop.h>
​
// this class will be used to analyze the events
class EventAnalyzer {
public:
    // class variables to store global information
    int global_counter                 = 0; // this will track how many events we processed
    Metavision::timestamp global_max_t = 0; // this will track the highest timestamp we processed
​
    // this function will be associated to the camera callback
    // it is used to compute statistics on the received events
    void analyze_events(const Metavision::EventCD *begin, const Metavision::EventCD *end) {
        std::cout << "----- New callback! -----" << std::endl;
​
        // time analysis
        // Note: events are ordered by timestamp in the callback, so the first event will have the lowest timestamp and
        // the last event will have the highest timestamp
        Metavision::timestamp min_t = begin->t;     // get the timestamp of the first event of this callback
        Metavision::timestamp max_t = (end - 1)->t; // get the timestamp of the last event of this callback
        global_max_t = max_t; // events are ordered by timestamp, so the current last event has the highest timestamp
​
        // counting analysis
        int counter = 0;
        for (const Metavision::EventCD *ev = begin; ev != end; ++ev) {
            ++counter; // increasing local counter
        }
        global_counter += counter; // increase global counter
​
        // report
        std::cout << "There were " << counter << " events in this callback" << std::endl;
        std::cout << "There were " << global_counter << " total events up to now." << std::endl;
        std::cout << "The current callback included events from " << min_t << " up to " << max_t << " microseconds."
                  << std::endl;
​
        std::cout << "----- End of the callback! -----" << std::endl;
    }
};
​
// main loop
int main(int argc, char *argv[]) {
    Metavision::Camera cam;       // create the camera
    EventAnalyzer event_analyzer; // create the event analyzer
​
    if (argc >= 2) {
        // if we passed a file path, open it
        cam = Metavision::Camera::from_file(argv[1]);
    } else {
        // open the first available camera
        cam = Metavision::Camera::from_first_available();
    }
​
    // to analyze the events, we add a callback that will be called periodically to give access to the latest events
    cam.cd().add_callback([&event_analyzer](const Metavision::EventCD *ev_begin, const Metavision::EventCD *ev_end) {
        event_analyzer.analyze_events(ev_begin, ev_end);
    });
​
    // to visualize the events, we will need to build frames and render them.
    // building frame will be done with a frame generator that will accumulate the events over time.
    // we need to provide it the camera resolution that we can retrieve from the camera instance
    int camera_width  = cam.geometry().width();
    int camera_height = cam.geometry().height();
​
    // we also need to choose an accumulation time and a frame rate (here of 20ms and 50 fps)
    const std::uint32_t acc = 20000;
    double fps              = 50;
​
    // now we can create our frame generator using previous variables
    auto frame_gen = Metavision::PeriodicFrameGenerationAlgorithm(camera_width, camera_height, acc, fps);
​
    // we add the callback that will pass the events to the frame generator
    cam.cd().add_callback([&](const Metavision::EventCD *begin, const Metavision::EventCD *end) {
        frame_gen.process_events(begin, end);
    });
​
    // to render the frames, we create a window using the Window class of the UI module
    Metavision::Window window("Metavision SDK Get Started", camera_width, camera_height,
                              Metavision::BaseWindow::RenderMode::BGR);
​
    // we set a callback on the windows to close it when the Escape or Q key is pressed
    window.set_keyboard_callback(
        [&window](Metavision::UIKeyEvent key, int scancode, Metavision::UIAction action, int mods) {
            if (action == Metavision::UIAction::RELEASE &&
                (key == Metavision::UIKeyEvent::KEY_ESCAPE || key == Metavision::UIKeyEvent::KEY_Q)) {
                window.set_close_flag();
            }
        });
​
    // we set a callback on the frame generator so that it calls the window object to display the generated frames
    frame_gen.set_output_callback([&](Metavision::timestamp, cv::Mat &frame) { window.show(frame); });
​
    // start the camera
    cam.start();
​
    // keep running until the camera is off, the recording is finished or the escape key was pressed
    while (cam.is_running() && !window.should_close()) {
        // we poll events (keyboard, mouse etc.) from the system with a 20ms sleep to avoid using 100% of a CPU's core
        // and we push them into the window where the callback on the escape key will ask the windows to close
        static constexpr std::int64_t kSleepPeriodMs = 20;
        Metavision::EventLoop::poll_and_dispatch(kSleepPeriodMs);
    }
​
    // the recording is finished or the user wants to quit, stop the camera.
    cam.stop();
​
    // print the global statistics
    const double length_in_seconds = event_analyzer.global_max_t / 1000000.0;
    std::cout << "There were " << event_analyzer.global_counter << " events in total." << std::endl;
    std::cout << "The total duration was " << length_in_seconds << " seconds." << std::endl;
    if (length_in_seconds >= 1) { // no need to print this statistics if the video was too short
        std::cout << "There were " << event_analyzer.global_counter / length_in_seconds
                  << " events per second on average." << std::endl;
    }
}

 

Metavision SDK是迄今为止最大的基于事件的视觉算法集。其高性能算法可通过api获得,辅以基于事件的视觉应用程序,已可投入生产

所提供的算法可以作为C++ api提供高效的应用程序和运行执行,也可以作为Python api获取对c++算法、机器学习管道等的访问。

 

 

开发高性能基于事件的视觉解决方案

32 个C++代码示例, 112 个Python类63 个算法, 47 个Python示例, 24个教程和17个现成的应用程序

主要特点
  • 大量的文档/代码示例/培训和学习材料可用
  • 本机在Linux和Windows上运行
  • 兼容Prophesee视觉系统和«由prophecy提供支持的»合作伙伴产品
  • 一个完整的API,允许您仅用短短几行代码即可探索基于事件的视觉的巨大潜力。
/**********************************************************************************************************************
 * Copyright (c) Prophesee S.A.                                                                                       *
 *                                                                                                                    *
 * Licensed under the Apache License, Version 2.0 (the "License");                                                    *
 * you may not use this file except in compliance with the License.                                                   *
 * You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0                                 *
 * Unless required by applicable law or agreed to in writing, software distributed under the License is distributed   *
 * on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.                      *
 * See the License for the specific language governing permissions and limitations under the License.                 *
 **********************************************************************************************************************/
​
// This code sample demonstrate how to use the Metavision C++ SDK. The goal of this sample is to create a simple event
// counter and displayer by introducing some basic concepts of the Metavision SDK.
​
#include <metavision/sdk/driver/camera.h>
#include <metavision/sdk/base/events/event_cd.h>
#include <metavision/sdk/core/algorithms/periodic_frame_generation_algorithm.h>
#include <metavision/sdk/ui/utils/window.h>
#include <metavision/sdk/ui/utils/event_loop.h>
​
// this class will be used to analyze the events
class EventAnalyzer {
public:
    // class variables to store global information
    int global_counter                 = 0; // this will track how many events we processed
    Metavision::timestamp global_max_t = 0; // this will track the highest timestamp we processed
​
    // this function will be associated to the camera callback
    // it is used to compute statistics on the received events
    void analyze_events(const Metavision::EventCD *begin, const Metavision::EventCD *end) {
        std::cout << "----- New callback! -----" << std::endl;
​
        // time analysis
        // Note: events are ordered by timestamp in the callback, so the first event will have the lowest timestamp and
        // the last event will have the highest timestamp
        Metavision::timestamp min_t = begin->t;     // get the timestamp of the first event of this callback
        Metavision::timestamp max_t = (end - 1)->t; // get the timestamp of the last event of this callback
        global_max_t = max_t; // events are ordered by timestamp, so the current last event has the highest timestamp
​
        // counting analysis
        int counter = 0;
        for (const Metavision::EventCD *ev = begin; ev != end; ++ev) {
            ++counter; // increasing local counter
        }
        global_counter += counter; // increase global counter
​
        // report
        std::cout << "There were " << counter << " events in this callback" << std::endl;
        std::cout << "There were " << global_counter << " total events up to now." << std::endl;
        std::cout << "The current callback included events from " << min_t << " up to " << max_t << " microseconds."
                  << std::endl;
​
        std::cout << "----- End of the callback! -----" << std::endl;
    }
};
​
// main loop
int main(int argc, char *argv[]) {
    Metavision::Camera cam;       // create the camera
    EventAnalyzer event_analyzer; // create the event analyzer
​
    if (argc >= 2) {
        // if we passed a file path, open it
        cam = Metavision::Camera::from_file(argv[1]);
    } else {
        // open the first available camera
        cam = Metavision::Camera::from_first_available();
    }
​
    // to analyze the events, we add a callback that will be called periodically to give access to the latest events
    cam.cd().add_callback([&event_analyzer](const Metavision::EventCD *ev_begin, const Metavision::EventCD *ev_end) {
        event_analyzer.analyze_events(ev_begin, ev_end);
    });
​
    // to visualize the events, we will need to build frames and render them.
    // building frame will be done with a frame generator that will accumulate the events over time.
    // we need to provide it the camera resolution that we can retrieve from the camera instance
    int camera_width  = cam.geometry().width();
    int camera_height = cam.geometry().height();
​
    // we also need to choose an accumulation time and a frame rate (here of 20ms and 50 fps)
    const std::uint32_t acc = 20000;
    double fps              = 50;
​
    // now we can create our frame generator using previous variables
    auto frame_gen = Metavision::PeriodicFrameGenerationAlgorithm(camera_width, camera_height, acc, fps);
​
    // we add the callback that will pass the events to the frame generator
    cam.cd().add_callback([&](const Metavision::EventCD *begin, const Metavision::EventCD *end) {
        frame_gen.process_events(begin, end);
    });
​
    // to render the frames, we create a window using the Window class of the UI module
    Metavision::Window window("Metavision SDK Get Started", camera_width, camera_height,
                              Metavision::BaseWindow::RenderMode::BGR);
​
    // we set a callback on the windows to close it when the Escape or Q key is pressed
    window.set_keyboard_callback(
        [&window](Metavision::UIKeyEvent key, int scancode, Metavision::UIAction action, int mods) {
            if (action == Metavision::UIAction::RELEASE &&
                (key == Metavision::UIKeyEvent::KEY_ESCAPE || key == Metavision::UIKeyEvent::KEY_Q)) {
                window.set_close_flag();
            }
        });
​
    // we set a callback on the frame generator so that it calls the window object to display the generated frames
    frame_gen.set_output_callback([&](Metavision::timestamp, cv::Mat &frame) { window.show(frame); });
​
    // start the camera
    cam.start();
​
    // keep running until the camera is off, the recording is finished or the escape key was pressed
    while (cam.is_running() && !window.should_close()) {
        // we poll events (keyboard, mouse etc.) from the system with a 20ms sleep to avoid using 100% of a CPU's core
        // and we push them into the window where the callback on the escape key will ask the windows to close
        static constexpr std::int64_t kSleepPeriodMs = 20;
        Metavision::EventLoop::poll_and_dispatch(kSleepPeriodMs);
    }
​
    // the recording is finished or the user wants to quit, stop the camera.
    cam.stop();
​
    // print the global statistics
    const double length_in_seconds = event_analyzer.global_max_t / 1000000.0;
    std::cout << "There were " << event_analyzer.global_counter << " events in total." << std::endl;
    std::cout << "The total duration was " << length_in_seconds << " seconds." << std::endl;
    if (length_in_seconds >= 1) { // no need to print this statistics if the video was too short
        std::cout << "There were " << event_analyzer.global_counter / length_in_seconds
                  << " events per second on average." << std::endl;
    }
}

 

硬件

购买兼容的硬件

由Prophesee 合作实现的Prophesee 评估套件封装传感器索尼IMX636ES能够完全兼容Metavision智能套件以及世纪方舟由Prophesee提供支持的SilkyEvCam

构建兼容硬件

通过广泛的合作计划,Prophesee 帮助视觉设备制造商构建其基于事件的视觉产品。

获取源代码,开发您自己的硬件,或将您的视觉系统与Metavision Intelligence集成。

常见问题解答

我需要买EVK开始吗?
评估工具包或基于事件的视觉设备并不一定是您探索新一代机器视觉的必需品。您可以从Metavision Studio 开始,初步尝试与我们提供的录像 进行交互。
支持哪些操作系统?
Metavision Intelligence套件支持Linux Ubuntu 18.04 / 20.04和Windows 10 64位(适用于PC X86-64架构)。有关其他操作系统兼容性,欢迎咨询我们或查看我们的开源项目OpenEB
我可以用提供的许可证做什么?
该套件是在商业许可下提供的,允许您免费下载、使用、构建甚至出售自己的商业应用程序。点击此处阅读许可协议
支持哪些基于事件的视觉硬件?
该软件套件与prophecy Metavision传感器评估套件兼容,还可与兼容的第三方产品共同运行。
会提供Python示例吗?
我们提供了python示例代码:https://docs.prophesee.ai/stable/samples.html

即刻免费下载

免费下载,免费制作,免费销售

我们的软件工具包可免费在Windows和Ubuntu PC上使用,免费期限无任何时间限制。其涵盖一个商业级许可证,使您能够利用Prophesee 基于事件的Metavision传感技术,设计和销售自身基于Metavision智能的产品