Patching VMware Tools in Fedora 18
Patching VMware Tools so HGFS (host-guest file system) will run in Fedora 18
There are several bugs in the current release of VMware Tools (VMwareTools-9.2.2-893683). Below are some patches and scripts to fix those. You can download these files from my GitHub repo:https://github.com/ebdevrepo/bin
People have also had good results with this patch: http://communities.vmware.com/message/2130250
(1) Kernel include file bug
There is currently a bug in the vmware-install.pl script. It will fail to find header files. To fix that, run this script [download] and then re-run vmware-install.pl:#!/bin/bash
KVER=`uname -r`
cp /usr/src/kernels/$KVER/include/generated/uapi/linux/version.h /lib/modules/$KVER/build/include/linux/
(2) VMCI driver bug
There is a second bug in the vmware-install.pl script. It will fail to compile the vmci driver. See these posts:
https://patchwork.kernel.org/patch/1963371/
http://communities.vmware.com/message/2182440#2182440
You know you are encountering this bug if you see these compile errors when running vmware-install.pl:
make[1]: Entering directory `/usr/src/kernels/3.8.1-201.fc18.x86_64'
CC [M] /tmp/modconfig-grfWex/vmci-only/linux/driver.o
/tmp/modconfig-grfWex/vmci-only/linux/driver.c:127:4: error: implicit declaration of function ‘__devexit_p’ [-Werror=implicit-function-declaration]
/tmp/modconfig-grfWex/vmci-only/linux/driver.c:127:4: error: initializer element is not constant
/tmp/modconfig-grfWex/vmci-only/linux/driver.c:127:4: error: (near initialization for ‘vmci_driver.remove’)
/tmp/modconfig-grfWex/vmci-only/linux/driver.c:1754:1: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘vmci_probe_device’
/tmp/modconfig-grfWex/vmci-only/linux/driver.c:1982:1: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘vmci_remove_device’
/tmp/modconfig-grfWex/vmci-only/linux/driver.c:119:12: warning: ‘vmci_probe_device’ used but never defined [enabled by default]
/tmp/modconfig-grfWex/vmci-only/linux/driver.c:121:13: warning: ‘vmci_remove_device’ used but never defined [enabled by default]
/tmp/modconfig-grfWex/vmci-only/linux/driver.c:2063:1: warning: ‘vmci_interrupt’ defined but not used [-Wunused-function]
/tmp/modconfig-grfWex/vmci-only/linux/driver.c:2137:1: warning: ‘vmci_interrupt_bm’ defined but not used [-Wunused-function]
/tmp/modconfig-grfWex/vmci-only/linux/driver.c:1717:1: warning: ‘vmci_enable_msix’ defined but not used [-Wunused-function]
cc1: some warnings being treated as errors
make[2]: *** [/tmp/modconfig-grfWex/vmci-only/linux/driver.o] Error 1
make[1]: *** [_module_/tmp/modconfig-grfWex/vmci-only] Error 2
make[1]: Leaving directory `/usr/src/kernels/3.8.1-201.fc18.x86_64'
make: *** [vmci.ko] Error 2
make: Leaving directory `/tmp/modconfig-grfWex/vmci-only'
The communication service is used in addition to the standard communication
between the guest and the host. The rest of the software provided by VMware
Tools is designed to work independently of this feature.
If you wish to have the VMCI feature, you can install the driver by running
vmware-config-tools.pl again after making sure that gcc, binutils, make and the
kernel sources for your running kernel are installed on your machine. These
packages are available on your distribution's installation CD.
To work around this, first install the patch utility:
sudo yum install patch
Create this vmware9.k3.8rc4.patch file [download] in your vmware-tools-distrib directory:
--- compat_mm.h.orig 2013-03-04 01:36:39.184925478 -0800
+++ compat_mm.h 2013-03-04 01:40:37.793728289 -0800
@@ -91,8 +91,9 @@
/*
* In 2.4.10, vmtruncate was changed from returning void to returning int.
+ * In 3.8.0, vmtruncate was removed
*/
-#if LINUX_VERSION_CODE < KERNEL_VERSION(2, 4, 10)
+#if LINUX_VERSION_CODE < KERNEL_VERSION(3, 8, 0)
#define compat_vmtruncate(inode, size) \
({ \
int result = 0; \
@@ -100,7 +101,16 @@
result; \
})
#else
-#define compat_vmtruncate(inode, size) vmtruncate(inode, size)
+#define compat_vmtruncate(inode, size) \
+({ \
+ result = inode_newsize_ok(inode, size); \
+ if (!result) \
+ { \
+ truncate_setsize(inode, size); \
+ } \
+ result; \
+})
+
#endif
Then run this script from your vmware-tools-distrib directory [download]. If it asks you if this is a reverse patch, select no (type the letter 'n').
#!/bin/bash
pushd lib/modules/source
if [[ ! -f vmci.tar.orig ]]
then
cp vmci.tar vmci.tar.orig
fi
rm -rf vmci-only
tar xf vmci.tar
pushd vmci-only
patch -p1 < ../../../../vmware9.k3.8rc4.patch
popd
tar cf vmci.tar vmci-only
rm -rf vmci-only
popd
You know you are encountering this bug if you see these compile errors when running vmware-install.pl:
CC [M] /tmp/modconfig-kxIA61/vmhgfs-only/inode.o
/tmp/modconfig-kxIA61/vmhgfs-only/inode.c: In function ‘HgfsTruncatePages’:
/tmp/modconfig-kxIA61/vmhgfs-only/inode.c:888:4: error: implicit declaration of function ‘vmtruncate’ [-Werror=implicit-function-declaration]
cc1: some warnings being treated as errors
make[2]: *** [/tmp/modconfig-kxIA61/vmhgfs-only/inode.o] Error 1
make[1]: *** [_module_/tmp/modconfig-kxIA61/vmhgfs-only] Error 2
make[1]: Leaving directory `/usr/src/kernels/3.8.1-201.fc18.x86_64'
make: *** [vmhgfs.ko] Error 2
make: Leaving directory `/tmp/modconfig-kxIA61/vmhgfs-only'
The filesystem driver (vmhgfs module) is used only for the shared folder
feature. The rest of the software provided by VMware Tools is designed to work
independently of this feature.
If you wish to have the shared folders feature, you can install the driver by
running vmware-config-tools.pl again after making sure that gcc, binutils, make
and the kernel sources for your running kernel are installed on your machine.
These packages are available on your distribution's installation CD.
Create this vmware9.compat_mm.patch file [download] in your vmware-tools-distrib directory:
--- ./compat_mm.h 2013-03-04 01:36:39.184925478 -0800
popd
** NOTE ** Any time there is a kernel update you will need to re-install VMware Tools to get hgfs working again (the ability for the VM to access files from the host OS).
https://patchwork.kernel.org/patch/1963371/
http://communities.vmware.com/message/2182440#2182440
You know you are encountering this bug if you see these compile errors when running vmware-install.pl:
make[1]: Entering directory `/usr/src/kernels/3.8.1-201.fc18.x86_64'
CC [M] /tmp/modconfig-grfWex/vmci-only/linux/driver.o
/tmp/modconfig-grfWex/vmci-only/linux/driver.c:127:4: error: implicit declaration of function ‘__devexit_p’ [-Werror=implicit-function-declaration]
/tmp/modconfig-grfWex/vmci-only/linux/driver.c:127:4: error: initializer element is not constant
/tmp/modconfig-grfWex/vmci-only/linux/driver.c:127:4: error: (near initialization for ‘vmci_driver.remove’)
/tmp/modconfig-grfWex/vmci-only/linux/driver.c:1754:1: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘vmci_probe_device’
/tmp/modconfig-grfWex/vmci-only/linux/driver.c:1982:1: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘vmci_remove_device’
/tmp/modconfig-grfWex/vmci-only/linux/driver.c:119:12: warning: ‘vmci_probe_device’ used but never defined [enabled by default]
/tmp/modconfig-grfWex/vmci-only/linux/driver.c:121:13: warning: ‘vmci_remove_device’ used but never defined [enabled by default]
/tmp/modconfig-grfWex/vmci-only/linux/driver.c:2063:1: warning: ‘vmci_interrupt’ defined but not used [-Wunused-function]
/tmp/modconfig-grfWex/vmci-only/linux/driver.c:2137:1: warning: ‘vmci_interrupt_bm’ defined but not used [-Wunused-function]
/tmp/modconfig-grfWex/vmci-only/linux/driver.c:1717:1: warning: ‘vmci_enable_msix’ defined but not used [-Wunused-function]
cc1: some warnings being treated as errors
make[2]: *** [/tmp/modconfig-grfWex/vmci-only/linux/driver.o] Error 1
make[1]: *** [_module_/tmp/modconfig-grfWex/vmci-only] Error 2
make[1]: Leaving directory `/usr/src/kernels/3.8.1-201.fc18.x86_64'
make: *** [vmci.ko] Error 2
make: Leaving directory `/tmp/modconfig-grfWex/vmci-only'
The communication service is used in addition to the standard communication
between the guest and the host. The rest of the software provided by VMware
Tools is designed to work independently of this feature.
If you wish to have the VMCI feature, you can install the driver by running
vmware-config-tools.pl again after making sure that gcc, binutils, make and the
kernel sources for your running kernel are installed on your machine. These
packages are available on your distribution's installation CD.
To work around this, first install the patch utility:
sudo yum install patch
Create this vmware9.k3.8rc4.patch file [download] in your vmware-tools-distrib directory:
--- compat_mm.h.orig 2013-03-04 01:36:39.184925478 -0800
+++ compat_mm.h 2013-03-04 01:40:37.793728289 -0800
@@ -91,8 +91,9 @@
/*
* In 2.4.10, vmtruncate was changed from returning void to returning int.
+ * In 3.8.0, vmtruncate was removed
*/
-#if LINUX_VERSION_CODE < KERNEL_VERSION(2, 4, 10)
+#if LINUX_VERSION_CODE < KERNEL_VERSION(3, 8, 0)
#define compat_vmtruncate(inode, size) \
({ \
int result = 0; \
@@ -100,7 +101,16 @@
result; \
})
#else
-#define compat_vmtruncate(inode, size) vmtruncate(inode, size)
+#define compat_vmtruncate(inode, size) \
+({ \
+ result = inode_newsize_ok(inode, size); \
+ if (!result) \
+ { \
+ truncate_setsize(inode, size); \
+ } \
+ result; \
+})
+
#endif
pushd lib/modules/source
if [[ ! -f vmci.tar.orig ]]
then
cp vmci.tar vmci.tar.orig
fi
rm -rf vmci-only
tar xf vmci.tar
pushd vmci-only
patch -p1 < ../../../../vmware9.k3.8rc4.patch
popd
tar cf vmci.tar vmci-only
rm -rf vmci-only
popd
(3) VMHGFS bug
You know you are encountering this bug if you see these compile errors when running vmware-install.pl:
CC [M] /tmp/modconfig-kxIA61/vmhgfs-only/inode.o
/tmp/modconfig-kxIA61/vmhgfs-only/inode.c: In function ‘HgfsTruncatePages’:
/tmp/modconfig-kxIA61/vmhgfs-only/inode.c:888:4: error: implicit declaration of function ‘vmtruncate’ [-Werror=implicit-function-declaration]
cc1: some warnings being treated as errors
make[2]: *** [/tmp/modconfig-kxIA61/vmhgfs-only/inode.o] Error 1
make[1]: *** [_module_/tmp/modconfig-kxIA61/vmhgfs-only] Error 2
make[1]: Leaving directory `/usr/src/kernels/3.8.1-201.fc18.x86_64'
make: *** [vmhgfs.ko] Error 2
make: Leaving directory `/tmp/modconfig-kxIA61/vmhgfs-only'
The filesystem driver (vmhgfs module) is used only for the shared folder
feature. The rest of the software provided by VMware Tools is designed to work
independently of this feature.
If you wish to have the shared folders feature, you can install the driver by
running vmware-config-tools.pl again after making sure that gcc, binutils, make
and the kernel sources for your running kernel are installed on your machine.
These packages are available on your distribution's installation CD.
Create this vmware9.compat_mm.patch file [download] in your vmware-tools-distrib directory:
--- ./compat_mm.h 2013-03-04 01:36:39.184925478 -0800
+++ ./compat_mm.h 2013-03-04 01:40:37.793728289 -0800
@@ -91,8 +91,9 @@
/*
* In 2.4.10, vmtruncate was changed from returning void to returning int.
+ * In 3.8.0, vmtruncate was removed
*/
-#if LINUX_VERSION_CODE < KERNEL_VERSION(2, 4, 10)
+#if LINUX_VERSION_CODE < KERNEL_VERSION(3, 8, 0)
#define compat_vmtruncate(inode, size) \
({ \
int result = 0; \
@@ -100,7 +101,16 @@
result; \
})
#else
-#define compat_vmtruncate(inode, size) vmtruncate(inode, size)
+#define compat_vmtruncate(inode, size) \
+({ \
+ result = inode_newsize_ok(inode, size); \
+ if (!result) \
+ { \
+ truncate_setsize(inode, size); \
+ } \
+ result; \
+})
+
#endif
#!/bin/bash
pushd lib/modules/source
if [[ ! -f vmhgfs.tar.orig ]]
then
cp vmhgfs.tar vmhgfs.tar.orig
fi
rm -rf vmhgfs-only
tar xf vmhgfs.tar
pushd vmhgfs-only/shared
patch -p1 < ../../../../../vmware9.compat_mm.patch
popd
tar cf vmhgfs.tar vmhgfs-only
rm -rf vmhgfs-only
I encountered bug 3 while installing Fedora 18 in my VM. Your steps helped me fix it. Thank you!
ReplyDeleteHELO!
ReplyDeleteFor me it raises :
patching file compat_mm.h
patch: **** malformed patch at line 14: @@ -100,7 +101,16 @@
Kernel 3.8.3.201 FC 18
Marfaru,
ReplyDeleteIt is likely just a cut-and-paste problem. Try downloading the patch directly from my GitHub repo:
https://raw.github.com/ebdevrepo/bin/master/vmware9.compat_mm.patch
Thanks! from me as well :)
ReplyDeleteI still get the hgfs compile error after patching...
ReplyDelete# uname -a
Linux localhost.localdomain 3.8.5-201.fc18.x86_64 #1 SMP Thu Mar 28 21:01:19 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
Any help appreciated...
Donnie - I just tried a fresh install of VMWare Tools on 3.8.5 and the bug fixes each worked fine. I did find a bug in the shell script that applies the patch, though. It breaks if you invoke it with a relative path. Maybe the patch didn't get applied. Try it again, watching to make sure that the patch doesn't fail.
ReplyDeleteI managed to encounter all of these but got here after using a similar solution to the first bug.
ReplyDeleteI am wondering why is it when I run the scripts from vmware-tools-distrib folder which i have in my home directory, I always get the error file not found. So instead of using the $PATH_DIR on both scripts, I used the absolute path ~/vmware-tools-distrib/. With this I managed to go through the second bug, but the vmhgfs still is giving me the same error which followed by the vmxnet throwing the similar error.
Any thoughts on this? Thanks!
I've fixed the path bugs, thanks for pointing those out.
DeleteDid you get any further with the patches, or is it still broken?
Yes I did, but I did it with another patch as well. For me it seems to work with both solutions.
DeleteHere was the other patch I used.
http://communities.vmware.com/message/2130250
Thanks you!
It worked for me too. Many thanks!
ReplyDeleteBrilliant, thanks!
ReplyDeleteThanks a lot!
ReplyDeletefor those that are experiencing the VMHGFS bug in Ubuntu 13.04 (bug number 3), the solution in this page works fine.
you just have to copy both files to the vmware tools distrib directory (/usr/lib/vmware-tools), and tweak the script by replacing the line
pushd lib/modules/source
with
pushd modules/source
After applying the script, vmware-config-tools.pl will compile fine.
I encountered these issues with vmci and vmhgfs while installing vmware-tools from VMWare Fusion 5.0.3 on Mac host and Ubuntu 13.0.4 (kernel 3.8.0.19-generic). I followed the instructions above and it solved my issues.
ReplyDeleteThanks a lot.
This comment has been removed by the author.
ReplyDeleteOk, I have realized that the lib is located in the dystro directory. But, instead of ‘vmtruncate’ I have got
ReplyDelete/tmp/modconfig-FiaFGk/vmhgfs-only/inode.c:1821:64: error: macro "hlist_for_each_entry" passed 4 arguments, but takes just 3
/tmp/modconfig-FiaFGk/vmhgfs-only/inode.c:1821:7: error: ‘hlist_for_each_entry’ undeclared (first use in this function)
In function ‘HgfsPermission’: error.
It is removed by one more patch, https://github.com/Sabayon/for-gentoo/blob/master/app-emulation/open-vm-tools-kmod/files/open-vm-tools-kmod-linux-3.9.patch
DeleteAll errors are gone, hgfs module is loaded but mount is not operational. I cannot access the host.
Ok, I had to re-enable the share in the VMWare. Thanks, now I have hgfs working.
ReplyDeleteGreat work Eric! Thank you for sharing this with the community!
ReplyDelete-Trevor
Excellent fix! Thank you so much for your help.
ReplyDeleteHGFS-patch works for Ubuntu 12.04 under VMware Workstation 9 on Windows 7 Pro.
ReplyDeleteThank you so much.