BundleFusion复现
源码地址： https://github.com/nonlinear1/BundleFusion_Ubuntu_V0
data： http://graphics.stanford.edu/projects/bundlefusion/#data
复现参考： https://blog.csdn.net/weixin_39152543/article/details/123021263
过程如下：
-
安装OpenGL
-
gcc5安装 GCC版本切换
-
找不到abs等库函数：网上需要切换到GCC5，一般不要换编译器，换完5后Eigen有问题，又换回gcc9;
-
去掉命名空间，找不到函数问题解决：原因：低版本的gcc还没有给abs等函数添加std命名空间，所以可以编译通过；修改改的文件路径:
*/BundleFusion_Ubuntu_V0-master/external/mLib/include/mLibDepthCamera.cpp
//namespace stb {
#define STB_IMAGE_IMPLEMENTATION
#include "ext-depthcamera/sensorData/stb_image.h"
#undef STB_IMAGE_IMPLEMENTATION
#define STB_IMAGE_WRITE_IMPLEMENTATION
#include "ext-depthcamera/sensorData/stb_image_write.h"
#undef STB_IMAGE_WRITE_IMPLEMENTATION
//}
- 安装openNI
安装包下载，网盘下载：http://pan.baidu.com/s/1hqHB10w 提取密码：wrmn
- 找不到-lXxf86vm
/usr/bin/ld: cannot find -lXxf86vm
/usr/bin/ld: cannot find -lXxf86vm
collect2: error: ld returned 1 exit status
解决办法：
$ sudo apt-get install libxxf86vm-dev
提示需要依赖libxxf86vm1
sudo apt-get install libxxf86vm1
提示版本不对,没有后缀build1，当前安装版本为有后缀build的版本
直接下载libxxf86vm的deb，https://launchpad.net/ubuntu/+source/libxxf86vm
选择amd64版本：
sudo dpkg -i libxxf86vm-dev_1.1.4-1build1_amd64.deb
至此，编译通过
- 运行出现问题：
cude error ：Source\SiftGPU\SiftPyramid.cpp(387) : cudaSafeCall()
Runtime API error 73: an illegal instruction was encountered.
排查发现是CmakeLists.txt中GPU设置问题：
set(CUDA_NVCC_FLAGS -arch=sm_80;-O3;-G;-g;-std=c++17)
查看自己 的GPU架构，修改为对应 的sm_*
查看网址：https://arnon.dk/matching-sm-architectures-arch-and-gencode-for-various-nvidia-cards/
- 编译报错：
__shfl_xor 等函数未定义
原因：
https://blog.csdn.net/kunhe0512/article/details/125492263
新版本的cuda弃用了这一类函数，需要使用它们的同步变体。
T __shfl_sync(unsigned mask, T var, int srcLane, int width=warpSize);
T __shfl_up_sync(unsigned mask, T var, unsigned int delta, int width=warpSize);
T __shfl_down_sync(unsigned mask, T var, unsigned int delta, int width=warpSize);
T __shfl_xor_sync(unsigned mask, T var, int laneMask, int width=warpSize);
替换即可。
同步变体参数和旧版本函数定义不同，多了第一个参数mask，暂时都使用0xffffffff。
即：
__shfl_xor(val, offset, 32)
改为：
__shfl_xor_sync(0xffffffff, val, offset, 32)
编译通过
- 运行黑屏，不报错
线程问题，参考：
https://github.com/niessner/BundleFusion/pull/56/files#
- size问题
SIFTImageManager.cpp:49: SIFTImageGPU& SIFTImageManager ::
createSIFTImageGPU(): Assertion 'm_SIFTImagesGPU.size() <
m_maxNumInages' failed Aborted (core dumped)
解决方法：修改zParametersBuildingDefault.txt文件中s_submapsize，改大一些，比如从10改为20；
- 复现mesh有空洞
原因应该是面片的数量设置；
解决方法：修改
zParametersDefault.txt文件中s_marchingCubesMaxNumTriangles，改大一些，比如从1900000改为6000000
- 添加realsense
参考： https://www.freesion.com/article/33731223871/
添加realsense实现类文件RealSenseSensor.cpp 以及 RealSenseSensor.h 文件。
由于 BundleFusion 工程中没有对于 RealSenseSensor 类的实现部分，为使 BundleFusion 工程能够使用 D435i 相机，需要编写其实现类（包括头文件和实现文件）。适用于 D415/D435/D435i 深度相机 的 RealSenseSensor 实现类可从 RealSenseSensor for D415/D435/D435i in BundleFusion 获取。下载解压后将头文件与实现文件拷贝至 “ BundleFusion-master/FriedLiver/Source ” 路径下。
- Realsense 有变量编译未定义
报错：
/usr/local/include/librealsense2/hpp/rs_types.hpp:90: undefined reference to `rs2_get_error_message'
/usr/local/include/librealsense2/hpp/rs_types.hpp:92: undefined reference to `rs2_get_failed_function'
/usr/local/include/librealsense2/hpp/rs_types.hpp:92: undefined reference to `rs2_get_failed_function'
原因：CmakeLists.txt未添加realsense依赖项，添加：
find_package(realsense2 2.29.0)
target_link_libraries(rs-hello-realsense ${DEPENDENCIES} ${realsense2_LIBRARY})
- realsense编译问题：
error: ‘RealSenseSensor’ does not name a type;
/home/ict/Downloads/learn/BundleFusion_Ubuntu_V0-master/FriedLiver/Source/FriedLiver.cpp添加头文件#include "RealSenseSensor.h"
为以下代码，解除注释：
if (GlobalAppState::get().s_sensorIdx == 6) {
#ifdef REAL_SENSE
// static RealSenseSensor s_realSenseSensor;
// return &s_realSenseSensor;
g_sensor = new RealSenseSensor;
return g_sensor;
#else
throw MLIB_EXCEPTION("Requires Real Sense SDK and REAL_SENSE macro");
#endif
}
- realsense可以运行，运行中断
INVALID FIRST CHUNK
<<< [Frame: 21 ] 99621 >>>
[ stop scanning and exit ]
- 制作realsense数据包
制作.sens数据集： https://www.pythonheidong.com/blog/article/496317/e5f17147665ca6cd5af2/
AttributeError: ‘dict_keys‘ object has no attribute ‘remove
改为list
dic1 = {'a':1, 'b':2, 'c':3}
# keys = list(dic1)
keys = dic1.keys()
keys.remove('b')
print(keys)
dic1 = {'a':1, 'b':2, 'c':3}
keys = list(dic1)
# keys = dic1.keys()
keys.remove('b')
print(keys)
按顺序执行脚本：
$ python3 bag2img.py
$ python3 associate.py depth-stamp.txt rgb-stamp.txt > associate.txt
$ python3 pose.py
[associate.py]
[bag2img.py]
[pose.py]
最终，改代码发现缺少freeimage.h的Linux库函数，无法继续。
- 尝试把数据直接喂进bundlefusion
open->openfilestream
SensorDataReader 中：
m_sensorData = new SensorData;
m_sensorData->loadFromFile(filename);
SensorData:: class CalibrationData
SensorData.h line:1239, m_frames[i].loadFromFile(in);
SensorData.h line:709, void loadFromFile(std::istream\& in) 加载深度图，RGB图像数据；
存储结构：
std::vector\<RGBDFrame> m_frames;
RGBDFrame成员变量：
UINT64 m_colorSizeBytes; //compressed byte size
UINT64 m_depthSizeBytes; //compressed byte size
unsigned char* m_colorCompressed; //compressed color data
unsigned char* m_depthCompressed; //compressed depth data
UINT64 m_timeStampColor; //time stamp color (convection: in microseconds)
UINT64 m_timeStampDepth; //time stamp depth (convention: in microseconds)
mat4f m_cameraToWorld; //camera trajectory: from current frame to base frame
.sens数据压缩格式：
m_colorCompressionType= 2
m_depthCompressionType= 1
enum COMPRESSION_TYPE_COLOR {
TYPE_COLOR_UNKNOWN = -1,
TYPE_RAW = 0,
TYPE_PNG = 1,
TYPE_JPEG = 2
};
enum COMPRESSION_TYPE_DEPTH {
TYPE_DEPTH_UNKNOWN = -1,
TYPE_RAW_USHORT = 0,
TYPE_ZLIB_USHORT = 1,
TYPE_OCCI_USHORT = 2
};
RGBDFrameCacheRead::FrameState成员变量：
vec3uc* m_colorFrame;
unsigned short* m_depthFrame;
RGBDFrameCacheRead：：成员变量 ：m_data
SensorData.h修改代码如下：
//设置图像尺寸
m_colorWidth = 1280;
m_colorHeight = 720;
m_depthWidth = 1280;
m_depthHeight = 720;
// m_depthShift = 0.001;
//设置数据压缩格式
m_colorCompressionType = COMPRESSION_TYPE_COLOR::TYPE_RAW;
m_depthCompressionType = COMPRESSION_TYPE_DEPTH::TYPE_RAW_USHORT;
std::cout << "m_colorCompressionType= " << m_colorCompressionType << std::endl; //4
std::cout << "m_depthCompressionType= " << m_depthCompressionType << std::endl; //4
std::cout << "m_depthShift= " << m_depthShift << std::endl; //4
UINT64 numFrames = 0;
in.read((char*)&numFrames, sizeof(UINT64));
m_frames.resize(numFrames);
//resize 帧数
m_frames.resize(800);
std::cout << "frame size = " << m_frames.size() << std::endl;
int num_img = 0;
for (size_t i = 0; i < m_frames.size(); i++) {
// std::cout << "i = " << i << std::endl;
// m_frames[i].loadFromFile(in);
//为成员变量赋值
//外参
m_frames[i].free();
float extrinsic_i[16] = {1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1};
ml::mat4f extrinsic_w(extrinsic_i);
m_frames[i].m_cameraToWorld = extrinsic_w;
//时间戳
m_frames[i].m_timeStampColor = 0;
m_frames[i].m_timeStampDepth = 0;
// m_frames[i].m_colorCompressed = (unsigned char*)std::malloc(m_colorSizeBytes);
std::string str_num = to_string(num_img);
while (strlen(str_num.c_str()) < 4)
{
str_num = "0" + str_num;
}
std::string rgb_path="/home/ict/Documents/result/frame-00" + str_num + ".color.jpg";
std::string dep_path="/home/ict/Documents/result/frame-00" + str_num + ".depth.png";
// FILE *fp_rgb, *fp_dep;
// int size_rgb, size_dep;
// //rgb图像的size和指针
// fp_rgb = fopen(rgb_path.c_str(),"rb+");
// fseek(fp_rgb, 0, SEEK_END);
// size_rgb = ftell(fp_rgb);
// m_frames[i].m_colorSizeBytes = size_rgb + 1;
// rewind(fp_rgb);
// char* data_rgb = (char *)malloc(sizeof(char) * (size_rgb + 1));
// fread(data_rgb, size_rgb, 1, fp_rgb);
// m_frames[i].m_colorCompressed = (unsigned char*)std::malloc(sizeof(char) * (size_rgb + 1));
// fread(m_frames[i].m_colorCompressed, size_rgb, 1, fp_rgb);
// fclose(fp_rgb);
// std::cout << "rgb_img size = " << m_frames[i].m_colorSizeBytes << std::endl;
//depth图像的size和指针
// fp_dep = fopen(dep_path.c_str(),"rb+");
// fseek(fp_dep, 0, SEEK_END);
// size_dep = ftell(fp_dep);
// m_frames[i].m_depthSizeBytes = size_dep + 1;
// rewind(fp_dep);
// char* data_dep = (char *)malloc(sizeof(char) * (size_dep + 1));
// fread(data_dep, size_dep, 1, fp_dep);
// m_frames[i].m_depthCompressed = (unsigned char*)std::malloc(sizeof(char) * (size_dep + 1));
// fread(m_frames[i].m_depthCompressed, size_dep, 1, fp_dep);
// fclose(fp_dep);
// boost::format fmt("./%s/%s.%s");
cv::Mat rgb_img = cv::imread(rgb_path.c_str());
m_frames[i].m_colorSizeBytes = rgb_img.total() * rgb_img.elemSize();
m_frames[i].m_colorCompressed = (unsigned char*)std::malloc(sizeof(char) * (m_frames[i].m_colorSizeBytes));
memcpy(m_frames[i].m_colorCompressed, rgb_img.ptr(), m_frames[i].m_colorSizeBytes);
std::cout << "rgb_img size = " << m_frames[i].m_colorSizeBytes << std::endl;
cv::Mat dep_img = cv::imread(dep_path.c_str(), -1);
m_frames[i].m_depthSizeBytes = dep_img.total() * dep_img.elemSize();
m_frames[i].m_depthCompressed = (unsigned char*)std::malloc(sizeof(char) * m_frames[i].m_depthSizeBytes);
memcpy(m_frames[i].m_depthCompressed, dep_img.ptr(), m_frames[i].m_depthSizeBytes);
//创建一个大小与文件大小、类型相同的Mat src，h为src.rows,w为src.cols
std::cout << "i = " << i << "dep_img size = " << m_frames[i].m_depthSizeBytes << std::endl;
num_img += 2;
}
Tsdf
https://gitcode.net/mirrors/andyzeng/tsdf-fusion-python?utm_source=csdn_github_accelerator