vmware-specific-specific 5.5.x and kernel 2.6.31
Posted on September 11th, 2009
You can follow any responses to this entry through the RSS 2.0 feed.Another kernel and another patch, this one fixes issues with vmnet and net_device_ops compat mode being removed. Have fun.

September 17th, 2009 at 10:15 am
works perfekt.
thanks!
September 23rd, 2009 at 5:34 pm
/usr/src/linux-headers-2.6.31-020631-generic/arch/x86/include/asm/msr-index.h:377:1: warning: this is the location of the previous definition
CC [M] /tmp/vmware-config0/vmmon-only/vmcore/compat.o
CC [M] /tmp/vmware-config0/vmmon-only/vmcore/moduleloop.o
LD [M] /tmp/vmware-config0/vmmon-only/vmmon.o
Building modules, stage 2.
MODPOST 1 modules
WARNING: /tmp/vmware-config0/vmmon-only/vmmon.o (.properties): unexpected non-allocatable section.
Did you forget to use “ax”/”aw” in a .S file?
Note that for example contains
section definitions for use in .S files.
WARNING: “init_mm” [/tmp/vmware-config0/vmmon-only/vmmon.ko] undefined!
CC /tmp/vmware-config0/vmmon-only/vmmon.mod.o
LD [M] /tmp/vmware-config0/vmmon-only/vmmon.ko
make[1]: Leaving directory `/usr/src/linux-headers-2.6.31-020631-generic’
cp -f vmmon.ko ./../vmmon.o
make: Leaving directory `/tmp/vmware-config0/vmmon-only’
Unable to make a vmmon module that can be loaded in the running kernel:
insmod: error inserting ‘/tmp/vmware-config0/vmmon.o’: -1 Unknown symbol in module
There is probably a slight difference in the kernel configuration between the
set of C header files you specified and your running kernel. You may want to
rebuild a kernel based on that directory, or specify another directory.
September 23rd, 2009 at 5:36 pm
shoulder, you need to apply the kernel patch to export init_mm and reboot with that kernel, first.
The patch is included in the tarball. you need the kernel sources to do that.
See the README file.
September 24th, 2009 at 2:04 am
kang
Thanks for your help.
i’ve download all thoes files from
http://kernel.ubuntu.com/~kernel-ppa/mainline/v2.6.31/
and rebuild the kernel after patch init_mm
it’s working now.
Thanks again.
Sincerly.
October 4th, 2009 at 5:21 pm
Kang, I have tried to install this on both SuSE 11.2 M7 and M8. I was not successful on either. Primarily cause I could not re-build the kernel with the patches. Swear I followed all the correct steps, but alas to no avail. Steps taken are on a WebPage on my site IF you are interested. The page is:
http://www.trcompu.com/Computers/LinNotes/NewSuSE-11-2.html
Scroll down a little and then click on the VMware jump. I have it working, with your fixes, in openSuSE 11.1 in both KDE3 and KDE4. Don’t want to move to 11.2 until I can get VMware to working. Thanks in advance and thanks for the past.
Chuck
October 9th, 2009 at 2:55 pm
I hate to admit it but I am still learning to Linux. I am trying to switch from Win>Lin but I still need Win so I wanted to use VM. I got the Beta Ubuntu 9.10. I got the above download and tried to install but got the errors. How do I apply the patch included? The web is confusing on this?
Thanks,
October 9th, 2009 at 4:24 pm
Hi RLJ,
Unfortunately you have to recompile and patch your kernel for this to work. This is not trivial. While opensource allow you to fix things like we do, it often involve some hand work.
(Well to be precise, it would be really fixed if I patched things not to use init-mm which the kernel needs a patch for, but I lack the time and interest
Anyways, please refer to Ubuntu support forums or documentation on how to make a custom kernel.
It will involve getting a ‘kernel source’ package, going probably into /usr/src/linux-2.6.xx… and then applying the patch by hand with:
cd /usr/src/linux-2.6.xx….
patch -p1 < /path/to/the/patch
check theres no rejects, and carry on with your kernel compilation (probably will involve a "make-kpkg" in ubuntu)
For more information see: https://help.ubuntu.com/community/Kernel/Compile
October 9th, 2009 at 4:28 pm
Chuck Taylor,
Unfortunately I cannot seem to be able to access your webpage.
I suppose the Suse kernel is a bit different and my patch has rejects.
You may be able to read the patch by hand (using your favorite text editor), and figure out the file to modify from it and modify it by hand.
Its not that difficult really
The file is:
arch/x86/kernel/init_task.c
and you should add:
EXPORT_UNUSED_SYMBOL(init_mm);
at the end probably. (or anywhere). Hard to say without seeing the error or the Susde modifications, but it will probably work!
October 9th, 2009 at 4:31 pm
Thanks, that’s what it is about….learning!
I’ll start reading and ‘playing’.
Thanks again
October 23rd, 2009 at 3:05 pm
sorry, but I cannot get the kernel to export that symbol, even with the patch applied. I’m using linux-2.6.31.5
tried in various ways – using EXPORT_SYMBOL instead of _UNUSED_, though that should not make a difference. I added the EXPORT to init_task.c, and as it did not work, I added it to mm/init_mm.c, also adding the includes in order to make the kernel compile without warnings.
init_mm in fact shows up as a “D” symbol in /usr/src/linux-2.6.31.5/System.map
however, after rebooting with the new kernel, a “cat /proc/kallsyms | grep init_mm” only shows a “T init_mmap….” – no init_mm there. consequently, building vmware fails. that’s not good
how should I proceed?
October 23rd, 2009 at 3:16 pm
Herp, do you have a distro specific kernel?
I’m on 2.6.31.5 “vanilla” using the included patch and it exports fine. My colleagues are also using it, with vanilla kernels.
By vanilla i mean straight off kernel.org
October 23rd, 2009 at 3:23 pm
hm, strange, it seems I had to “make distclean” first, a simple “make bzImage” would not be sufficient. *sigh* oh my…
October 23rd, 2009 at 3:45 pm
still not 100% working. vmnet fails:
# modprobe vmnet
# dmesg
….
————[ cut here ]————
WARNING: at fs/proc/generic.c:590 proc_register+0×116/0×142()
Hardware name:
proc_dir_entry ‘/proc/vmnet’ already registered
Modules linked in: vmnet(P+) vmmon(P) sg snd_pcm_oss snd_pcm snd_timer snd_page_alloc snd_mixer_oss snd soundcore sr_mod cdrom nfsd lockd sunrpc exportfs e1000e 8139too nf_conntrack_ipv4 nf_defrag_ipv4 xt_state nf_conntrack xt_multiport xt_tcpudp iptable_filter ip_tables x_tables usbhid uhci_hcd usbcore nls_base tun nvram rtc hwmon_vid hwmon [last unloaded: vmnet]
Pid: 3323, comm: modprobe Tainted: P W 2.6.31.5 #2
Call Trace:
[] ? warn_slowpath_common+0×5d/0×70
[] ? warn_slowpath_fmt+0×26/0×2a
[] ? proc_register+0×116/0×142
[] ? create_proc_entry+0×6d/0×80
[] ? VNetProc_MakeEntry+0×18/0×28 [vmnet]
[] ? init_module+0×0/0xce [vmnet]
[] ? VNetProc_Init+0×16/0×1b [vmnet]
[] ? init_module+0×7/0xce [vmnet]
[] ? init_module+0×0/0xce [vmnet]
[] ? do_one_initcall+0×44/0×120
[] ? sys_init_module+0xa7/0×1b3
[] ? sysenter_do_call+0×12/0×22
—[ end trace 9fcc6f77dc3674bb ]—
October 23rd, 2009 at 3:46 pm
kang,
right, I’m using vanilla kernel. Distro = none – Linux from scratch
October 23rd, 2009 at 3:52 pm
ad “/proc/vmnet” already existing:
# rmmod vmnet
# lsmod | grep vmnet
(… nothing appears)
# ls -ld /proc/vmnet
dr-xr-xr-x … /proc/vmnet
okay then – I’m gonna reboot the system, maybe /proc/vmnet is a left-over of a modprobe gone nuts.
Looks good. /proc/vmnet exist, vmnet-module has a use count of 13 – YES! vmware boots.
well, everything I need works
thanks for the patch!
October 23rd, 2009 at 3:56 pm
by the way, with vmmon loaded, kallsyms shows:
# cat /proc/kallsyms | grep init_mm
c12d4d66 T init_mmap_min_addr
c12a7ac0 u init_mm [vmmon]
what does the “u” stand for?
November 3rd, 2009 at 4:22 pm
Sorry, but it did not work with kernel 2.6.31.14
…
Lot of errors: most …MSC_VER” … is not defined
…
Execution aborted.
Thanks for a short answer !!!
Johann
November 3rd, 2009 at 4:44 pm
Short answer: sorry, theres no kernel 2.6.31.14 yet…
i’m not sure how you get the MSC_VER errors, but probably you are missing a build environement. Verify you got gcc, g++, and all development libraries installed for your distribution.
November 4th, 2009 at 1:49 pm
Thanks !
gcc and g++ environment is completly installed. I got the MSC_VER errors while running the runme.pl
Kernel 2.6.31-14-generic is now in ubuntu karmic 9.10… Hope there will be a script for this kernel in future. Many thanks for your work and help !!!
November 4th, 2009 at 1:55 pm
Sorry Johann, but I’m currently running Ubuntu Karmic and 2.6.31 without any trouble with this vmware patch.
I would probably need more details (like complete error output) to know where the error is coming from.
ps: check you have libc6-dev installed
November 6th, 2009 at 8:08 pm
I am still hitting a wall on this. I can not find any good EASY TO FOLLOW instructions on patching the kernel to be able to utilize VM on 9.10.
Can someone provide a clear step by step on the process of patching so I can run the script on 9.10?
Thanks,
November 11th, 2009 at 8:27 pm
I can not install VMware Server-1.0.8 on my Ubuntu Karmic kernel 2.6.31.
I utilize the patch vmware-update-2.6.31-5.5.9.tar.bz2.
This is the output erro:
WARNING: /tmp/vmware-config5/vmmon-only/vmmon.o (.properties): unexpected non-allocatable section.
Did you forget to use “ax”/”aw” in a .S file?
Note that for example contains
section definitions for use in .S files.
WARNING: “init_mm” [/tmp/vmware-config5/vmmon-only/vmmon.ko] undefined!
CC /tmp/vmware-config5/vmmon-only/vmmon.mod.o
LD [M] /tmp/vmware-config5/vmmon-only/vmmon.ko
make[1]: Leaving directory `/usr/src/linux-headers-2.6.31-14-generic-pae’
cp -f vmmon.ko ./../vmmon.o
make: Leaving directory `/tmp/vmware-config5/vmmon-only’
Unable to make a vmmon module that can be loaded in the running kernel:
insmod: error inserting ‘/tmp/vmware-config5/vmmon.o’: -1 Unknown symbol in module
There is probably a slight difference in the kernel configuration between the
set of C header files you specified and your running kernel. You may want to
rebuild a kernel based on that directory, or specify another directory.
For more information on how to troubleshoot module-related problems, please
visit our Web site at “http://www.vmware.com/download/modules/modules.html” and
“http://www.vmware.com/support/reference/linux/prebuilt_modules_linux.html”.
Execution aborted.
Somebody help me?
November 12th, 2009 at 7:44 pm
Got the same error as Mercurion.
Kubuntu 9.10, kernel 2.6.31-15-generic, VMware Server ***1.0.10***.
November 16th, 2009 at 10:55 pm
same as José Queiroz.
thanks anyway for the job done.
November 17th, 2009 at 2:17 pm
you kernel is patched incorrectly. did you really patch *and* install it ?
November 18th, 2009 at 9:53 pm
Hello again,
I have a report. OpenSuSE 11.2 Final Release kernel 2.6.31.5-0.1 can be modified to use your changes. Make the change to init_task.c then:
make cloneconfig
make
make modules_install
make install
You may or may not experience a focus problem. I set my VMware up for a 1114×768 screen. However, only the upper left 640×480 had focus. The Lord opened my eyes and had me click on the Full Screen option under the View Menu. In the Full Screen I had full focus with the mouse. Hitting Ctl-Alt takes you back to the smaller screen AND it will have full focus. Problem is you must do this every time you startup VMware. Also just learned that the “Separate Cursor Control” keys AND the Page Up / Page Down keys do not work. You must use the dual purpose keys in the Numeric Keypad. -BUT- it is useable.
Kang – Try my site again if you have time. There was a time period when I also could not get to yours. Mine is at:
http://www.trcompu.com/Computers/LinNotes/NewSuSE-11-2.html
Not as well organized as yours but you may find some interesting reading.
Thanks for your initial efforts,
Chuck
November 21st, 2009 at 6:20 pm
[...] http://www.insecure.ws/2009/09/11/vmware-specific-specific-5-5-x-and-kernel-2-6-31Share this on del.icio.usAdd this to Google BookmarksShare this on FacebookTweet This!Submit this to NetvibesPost this to IdenticaPost this to MySpaceShare this on TechnoratiShare this on FriendFeed No hay entradas relacionadas. [...]
November 25th, 2009 at 8:26 pm
Is there any work around without compiling the Kernel for VMware 1.x ? I don’t want to play with my production systems by compiling the new Kernel and it’s takes ages to compile..
Thanks in Advance
~Sandesh
November 25th, 2009 at 8:43 pm
Sandesh,
some kernels already do have the extra symbol exported (especially “entreprise” distributions) for this kind of patches, so you may try if it works.
Otherwise, you need to actually re-code part of the vmware driver to avoid using the said system call (init-mm) which takes some time and thus I haven’t done.
I use this quick patch to be able to run old vmware versions when necessary, but this is not my main virtualization solution.
I am not aware of any other patch that would have the init-mm functions recoded.
December 3rd, 2009 at 7:43 pm
:-/
does not work for 2.6.32 … what a pitty
December 3rd, 2009 at 7:44 pm
ill post for 2.6.32 tomorrow probably
not sure if i’ll go for 2.6.33 etc.. VirtualBox is very good.
January 13th, 2010 at 3:42 am
Kang,
I have done some searching and found that the init_mm reference is pulled in by the pgd_offset_k macro used in about 3 places in the vmmon_only code for vs 1.0.10. I tested the thoery by changing the macro in my kernel headers
/usr/src/kernels/2.6.31.9-174.fc12.x86_64/arch/x86/include/asm/pgtable.h
from
#define pgd_offset_k(address) pgd_offset(&init_mm, (address))
to
#define pgd_offset_k(address) pgd_offset(current->mm, (address))
This gave me a successful compile on fedora12. I now need to work out some nice new permission problems as I am using ldap auth and the console doesn’t seem to cope.
January 13th, 2010 at 4:27 am
Ok, that was wrong. Compiles fine. Module loads fine. System needed reboot to recover from starting a VM. Maybe someone with better kernel-foo than me can clean up that idea.
February 2nd, 2010 at 11:15 pm
Thanks for your work!
I have modified this patch slightly to get it to work on my Fedora 12 (2.6.31) installation. Your mileage may vary.
The changes are in three files in the tarballs:
vmmon.tar sources:
diff -Naur ./vmmon-only/include/compat_pgtable.h ./vmmon-only.new/include/compat_pgtable.h
— ./vmmon-only/include/compat_pgtable.h 2007-07-22 07:21:17.000000000 -0400
+++ ./vmmon-only.new/include/compat_pgtable.h 2010-02-02 15:54:55.000000000 -0500
@@ -72,7 +72,7 @@
#endif
-#ifdef pgd_offset_k
+#if (LINUX_VERSION_CODE < KERNEL_VERSION(2, 6, 25)) && defined(pgd_offset_k)
# define compat_pgd_offset_k(mm, address) pgd_offset_k(address)
#else
# define compat_pgd_offset_k(mm, address) pgd_offset(mm, address)
diff -Naur ./vmmon-only/linux/hostif.c ./vmmon-only.new/linux/hostif.c
— ./vmmon-only/linux/hostif.c 2009-04-21 10:47:54.000000000 -0400
+++ ./vmmon-only.new/linux/hostif.c 2010-02-02 15:56:33.000000000 -0500
@@ -564,6 +564,7 @@
static void
DoClearNXBit(VA vaddr)
{
+#if LINUX_VERSION_CODE = 2.6.25 */
+ int level;
+ pte_t *pte = lookup_address(vaddr & PAGE_MASK, &level);
+#endif /* linux version */
if (pte_val(*pte) & _PAGE_NX) {
/* pte_val() is not lvalue on x86 PAE */
#ifdef CONFIG_X86_PAE
vmnet.tar sources:
diff -Naur ./vmnet-only/compat_pgtable.h ./vmnet-only.new/compat_pgtable.h
— ./vmnet-only/compat_pgtable.h 2007-11-28 05:25:06.000000000 -0500
+++ ./vmnet-only.new/compat_pgtable.h 2010-02-02 15:58:37.000000000 -0500
@@ -76,7 +76,7 @@
#endif
-#ifdef pgd_offset_k
+#if (LINUX_VERSION_CODE < KERNEL_VERSION(2, 6, 25)) && defined(pgd_offset_k)
# define compat_pgd_offset_k(mm, address) pgd_offset_k(address)
#else
# define compat_pgd_offset_k(mm, address) pgd_offset(mm, address)
February 2nd, 2010 at 11:25 pm
Oh yeah – forgot to indicate that the patch above didn’t require a recompilation of the kernel to re-export init_mm – just apply the patch to the untarred tarballs, retar ‘em and vmware-config.pl away!
February 11th, 2010 at 11:02 pm
Hi,
I applied the patch to the files you mention here(run vmware.install.pl, then ./runme.pl on update directory and then vmware-config.pl), but it abort after asking about the bridged interface (i have 2 ), this is just the final lines:
tmp/vmware-config2/vmnet-only/vm_atomic.h:1608:7: warning: “_MSC_VER” is not defined
/tmp/vmware-config2/vmnet-only/vm_atomic.h:1741:7: warning: “_MSC_VER” is not defined
{standard input}: Assembler messages:
{standard input}:6426: Error: can’t resolve `.LVL257′ {*UND* section} – `.Ltext0′ {.text section}
make[4]: *** [/tmp/vmware-config2/vmnet-only/driver.o] Error 1
make[3]: *** [_module_/tmp/vmware-config2/vmnet-only] Error 2
make[2]: *** [sub-make] Error 2
make[1]: *** [all] Error 2
make[1]: se sale del directorio `/usr/src/linux-2.6.31.12-0.1-obj/x86_64/desktop’
make: *** [vmnet.ko] Error 2
make: se sale del directorio `/tmp/vmware-config2/vmnet-only’
Unable to build the vmnet module.
For more information on how to troubleshoot module-related problems, please
visit our Web site at “http://www.vmware.com/download/modules/modules.html” and
“http://www.vmware.com/support/reference/linux/prebuilt_modules_linux.html”.
Execution aborted.
i have opensSuSE 11.2 with kernel 2.6.31.12-0.1-desktop
do you have any idea to try about?
February 26th, 2010 at 1:21 am
Finally working!!! on:
openSuSE 11.2 x64 with kernel 2.6.31.12-0.1-desktop (fresh updated) and
openSuSE 11.2 x32 with kernel 2.6.31.12-0.1-default (fresh updated)
Thanks to Kang suggestions and work, i made a little modifications to the second file that Kang shows,
with this (has Kang says) no needed to recompile kernel
diff -Naur vmware-update-2.6.31-5.5.9/vmmon-only/linux/hostif.c vmware-update-2.6.31-5.5.9-ok/vmmon-only/linux/hostif.c
— vmware-update-2.6.31-5.5.9/vmmon-only/linux/hostif.c 2009-04-21 08:47:54.000000000 -0600
+++ vmware-update-2.6.31-5.5.9-ok/vmmon-only/linux/hostif.c 2010-02-16 11:44:20.000000000 -0700
@@ -564,16 +564,10 @@
static void
DoClearNXBit(VA vaddr)
{
- int ptemap;
- pgd_t *pgd = pgd_offset_k(vaddr);
- pmd_t *pmd = pmd_offset_map(pgd, vaddr);
- pte_t *pte;
-
- if ((ptemap = pmd_val(*pmd) & _PAGE_PSE) != 0) {
- pte = (pte_t*)pmd;
- } else {
- pte = pte_offset_map(pmd, vaddr);
- }
+#if LINUX_VERSION_CODE = 2.6.25
+ int level;
+ pte_t *pte = lookup_address(vaddr & PAGE_MASK, &level);
+#endif /* linux version */
if (pte_val(*pte) & _PAGE_NX) {
/* pte_val() is not lvalue on x86 PAE */
#ifdef CONFIG_X86_PAE